Hi, lumenauts! Astrograph team is here with some project updates.
In-memory catchup
We've been focused primarily on Astrologer development recently, regarding using the pretty new stellar-core feature, called "in-memory catchup". Here's why it's important for our project.
Remind you that Astrologer is a tool for ingesting Stellar network ledgers into ElasticSearch indices. Since we would like to have past historical data (the whole history ideally), we have to obtain this history in the stellar-core database first. It is a problem because we have to first catch up with a huge number of ledgers and then keep them somewhere on disk to experiment with Astrologer. And, well, you should have stellar-core and database running somewhere 🙂 All of it produced overhead and additional complexity. And it can be expensive if you work in the cloud.
Some time ago stellar-core team added a --run-in-memory flag, which enables the core to apply ledgers without hitting the database and stream their meta to the pipe. We tried to incorporate it in the Astrologer, but it turned out that the Horizon team leverages this feature too in their [exp/ingest package](https://github.com/stellar/go/blob/master/exp/ingest/ledgerbackend/captive_core_backend.go) (kudos!). So we, in turn, used this package in Astrologer. This package operates stellar-core executable under the hood, so we can get rid of a separate stellar-core instance to ingest ledgers at all. Less operational complexity, how cool is that?
Pubnet endpoint
Using this new approach, we relaunched the pubnet instance of Astrograph — https://pubnet.astrograph.io. We've ingested the last three months' history for now, and we're ingesting every new ledger closed. We also switched Astrograph landing page (yep, we have one!) to the pubnet.
We're excited about this new stuff and will be happy to receive any feedback from you! You can tell us what you think here in replies, or on the dedicated public Keybase chat. An issue on the Github is the way too!
