Abstract Web is an elaborate, highly configurable web automation framework that lets users collect publicly available data via an intuitive CLI.

Features:

• Crawl websites
• Extract data from web pages
• Request APIs
• (Mass-)Download files
• Perform actions at webpages (e.g., click a button)
• State/cookie management (e.g., for logins)
• and much more (custom tasks)

Notably, it is able to abstract away (hence the name) the complexity of these features by providing pre-defined models of web pages.