Scraping public GoFundMe donation data

The crowdfunding platform GoFundMe currently makes the most recent 1000 donations to each fundraiser publicly available via a scrollable list. (Private payment-related information is not available, but a user’s public display name, as well as the amount and time of the donation, are.) Modern web platforms such as GoFundMe all work basically the same way — the user interface communicates with the site’s backend via REST (REpresentational State Transfer) web services. It is often possible to scrape publicly available user-generated content, such as a user’s social media posts or a list of fundraiser donations, by calling these web services directly. Here’s an overview of how to use this technique to write a simple GoFundMe donation scraper in Python.since Twitter is reinstating antivax accounts anyway, Alex Berenson’s fundraiser was likely a complete waste of time and money, but I digressWe can use the built-in developer tools included with most major browsers to peek under the hood and figure out how the web services underlying a given platform, such as GoFundMe, work. In Google Chrome (shown above), this is done by opening the “Developer Tools” under the “Developer” section of the “View” menu and clicking the “Network” tab; other browsers have similar features.Here we can inspect the requests sent to and the responses returned by the platform’s underlying web services. Generally, the responses will be in JSON (JavaScript Object Notation) format, which is easily parsed by most modern programming languages, including Python. In the case of GoFundMe, the donation list is retrieved…Scraping public GoFundMe donation data

Leave a Reply

Your email address will not be published. Required fields are marked *