In this category, I group all my posts about programming. I write mostly about Python, but am also curious to explore more general programming topics and venture into other programming languages.
In this second part, I present a practical example of how to write code for web scraping in a functional programming style. The example illustrates how to use immutable data structures, functions and Python’s @functools.singledispatch decorator to build a resilient data collection pipeline for web scraping.
For a long time, I have associated web scraping projects with a heavy dose of object-oriented programming. Python developers might be familiar with classes of spiders similar to those used in the web scraping framework Scrapy. In this two-part article, I present a way to write code for web scraping in a functional programming style. In this first part, I explain what I mean by that and what kind of advantages I see with that approach.
Special methods do not only power Python’s built-in objects. They can be used to give custom classes the same look and feel. In this post, I illustrate the use of special methods by defining a simple class of polynomials.