Update cookies preferences

E-book: Practical Web Scraping for Data Science: Best Practices and Examples with Python

3.64/5 (25 ratings by Goodreads)
  • Format: EPUB+DRM
  • Pub. Date: 18-Apr-2018
  • Publisher: APress
  • Language: eng
  • ISBN-13: 9781484235829
  • Format - EPUB+DRM
  • Price: 80,26 €*
  • * the price is final i.e. no additional discount will apply
  • Add to basket
  • Add to Wishlist
  • This ebook is for personal use only. E-Books are non-refundable.
  • Format: EPUB+DRM
  • Pub. Date: 18-Apr-2018
  • Publisher: APress
  • Language: eng
  • ISBN-13: 9781484235829

DRM restrictions

  • Copying (copy/paste):

    not allowed

  • Printing:

    not allowed

  • Usage:

    Digital Rights Management (DRM)
    The publisher has supplied this book in encrypted form, which means that you need to install free software in order to unlock and read it.  To read this e-book you have to create Adobe ID More info here. Ebook can be read and downloaded up to 6 devices (single user with the same Adobe ID).

    Required software
    To read this ebook on a mobile device (phone or tablet) you'll need to install this free app: PocketBook Reader (iOS / Android)

    To download and read this eBook on a PC or Mac you need Adobe Digital Editions (This is a free app specially developed for eBooks. It's not the same as Adobe Reader, which you probably already have on your computer.)

    You can't read this ebook with Amazon Kindle

This book provides a complete and modern guide to web scraping, using Python as the programming language, without glossing over important details or best practices. Written with a data science audience in mind, the book explores both scraping and the larger context of web technologies in which it operates, to ensure full understanding. The authors recommend web scraping as a powerful tool for any data scientist’s arsenal, as many data science projects start by obtaining an appropriate data set.

Starting with a brief overview on scraping and real-life use cases, the authors explore the core concepts of HTTP, HTML, and CSS to provide a solid foundation. Along with a quick Python primer, they cover Selenium for JavaScript-heavy sites, and web crawling in detail. The book finishes with a recap of best practices and a collection of examples that bring together everything you've learned and illustrate various data science use cases.

What You'll Learn

  • Leverage well-established best practices and commonly-used Python packages
  • Handle today's web, including JavaScript, cookies, and common web scraping mitigation techniques
  • Understand the managerial and legal concerns regarding web scraping
Who This Book is For

A data science oriented audience that is probably already familiar with Python or another programming language or analytical toolkit (R, SAS, SPSS, etc). Students or instructors in university courses may also benefit. Readers unfamiliar with Python will appreciate a quick Python primer in chapter 1 to catch up with the basics and provide pointers to other guides as well.
About the Authors ix
About the Technical Reviewer xi
Introduction xiii
Part I Web Scraping Basics
1(78)
Chapter 1 Introduction
3(22)
1.1 What Is Web Scraping?
3(5)
1.1.1 Why Web Scraping for Data Science?
4(1)
1.1.2 Who Is Using Web Scraping?
5(3)
1.2 Getting Ready
8(17)
1.2.1 Setting Up
8(1)
1.2.2 A Quick Python Primer
9(16)
Chapter 2 The Web Speaks HTTP
25(24)
2.1 The Magic of Networking
25(3)
2.2 The Hypertext Transfer Protocol: HTTP
28(6)
2.3 HTTP in Python: The Requests Library
34(5)
2.4 Query Strings: URLs with Parameters
39(10)
Chapter 3 Stirring the HTML and CSS Soup
49(30)
3.1 Hypertext Markup Language: HTML
49(2)
3.2 Using Your Browser as a Development Tool
51(5)
3.3 Cascading Style Sheets: CSS
56(5)
3.4 The Beautiful Soup Library
61(11)
3.5 More on Beautiful Soup
72(7)
Part II Advanced Web Scraping
79(94)
Chapter 4 Delving Deeper in HTTP
81(46)
4.1 Working with Forms and POST Requests
81(16)
4.2 Other HTTP Request Methods
97(3)
4.3 More on Headers
100(8)
4.4 Dealing with Cookies
108(11)
4.5 Using Sessions with Requests
119(2)
4.6 Binary, JSON, and Other Forms of Content
121(6)
Chapter 5 Dealing with JavaScript
127(28)
5.1 What Is JavaScript?
127(1)
5.2 Scraping JavaScript
128(6)
5.3 Scraping with Selenium
134(14)
5.4 More on Selenium
148(7)
Chapter 6 From Web Scraping to Web Crawling
155(18)
6.1 What Is Web Crawling?
155(3)
6.2 Web Crawling in Python
158(3)
6.3 Storing Results in a Database
161(12)
Part III Managerial Concerns and Best Practices
173(126)
Chapter 7 Managerial and Legal Concerns
175(12)
7.1 The Data Science Process
175(4)
7.2 Where Does Web Scraping Fit In?
179(2)
7.3 Legal Concerns
181(6)
Chapter 8 Closing Topics
187(10)
8.1 Other Tools
187(6)
8.1.1 Alternative Python Libraries
187(1)
8.1.2 Scrapy
188(1)
8.1.3 Caching
188(1)
8.1.4 Proxy Servers
189(1)
8.1.5 Scraping in Other Programming Languages
190(1)
8.1.6 Command-Line Tools
191(1)
8.1.7 Graphical Scraping Tools
191(2)
8.2 Best Practices and Tips
193(4)
Chapter 9 Examples
197(102)
9.1 Scraping Hacker News
199(2)
9.2 Using the Hacker News API
201(1)
9.3 Quotes to Scrape
202(4)
9.4 Books to Scrape
206(3)
9.5 Scraping GitHub Stars
209(5)
9.6 Scraping Mortgage Rates
214(6)
9.7 Scraping and Visualizing IMDB Ratings
220(2)
9.8 Scraping IATA Airline Information
222(6)
9.9 Scraping and Analyzing Web Forum Interactions
228(9)
9.10 Collecting and Clustering a Fashion Data Set
237(4)
9.11 Sentiment Analysis of Scraped Amazon Reviews
241(11)
9.12 Scraping and Analyzing News Articles
252(19)
9.13 Scraping and Analyzing a Wikipedia Graph
271(7)
9.14 Scraping and Visualizing a Board Members Graph
278(3)
9.15 Breaking CAPTCHA's Using Deep Learning
281(18)
Index 299
Seppe vanden Broucke is an assistant professor of data and process science at the Faculty of Economics and Business, KU Leuven, Belgium. His research interests include business data mining and analytics, machine learning, process management, and process mining. His work has been published in well-known international journals and presented at top conferences. Seppes teaching includes Advanced Analytics, Big Data and Information Management courses. He also frequently teaches for industry and business audiences. Besides work, Seppe enjoys travelling, reading (Murakami to Bukowski to Asimov), listening to music (Booka Shade to Miles Davis to Claude Debussy), watching movies and series (less so these days due to a lack of time), gaming, and keeping up with the news.

Bart Baesens is a professor of big data and analytics at KU Leuven, Belgium, and a lecturer at the University of Southampton, United Kingdom. He has done extensive research on big data and analytics, credit risk modeling, fraud detection and marketing analytics. Bart has written more than 200 scientific papers and several books. Besides enjoying time with his family, he is also a diehard Club Brugge soccer fan. Bart is a foodie and amateur cook. He loves drinking a good glass of wine (his favorites are white Viognier or red Cabernet Sauvignon) either in his wine cellar or when overlooking the authentic red English phone booth in his garden. Bart loves traveling and is fascinated by World War I and reads many books on the topic.