Aggregation Framework
- browse_activity In Progress
- code Scala
- tag json4s, Selenium
- file_save GitHub
A Swiss-army knife data scraping and processing tool written in Scala.
- Quickly build HTTP requests for a variety of data formats and APIs.
- Parse common data formats such as XML, HTML, JSON, and more.
- Push your aggregated data to Kafka topics for reliable realtime storage, or other datastores (such as Postgres or Hadoop)
- Write your own collectors for non-standard data formats.