cptlobster.dev

⇐ Back to all projects

Aggregation Framework

  • browse_activity In Progress
  • code Scala
  • tag json4s, Selenium
  • file_save GitHub

A Swiss-army knife data scraping and processing tool written in Scala.

  • Quickly build HTTP requests for a variety of data formats and APIs.
  • Parse common data formats such as XML, HTML, JSON, and more.
  • Push your aggregated data to Kafka topics for reliable realtime storage, or other datastores (such as Postgres or Hadoop)
  • Write your own collectors for non-standard data formats.

POS (80, 80)
ELEV 0.995 KM