https://astrograph.io Summary: Astrograph is a framework that lowers the entry threshold to the Stellar ecosystem and makes it possible to develop Stellar-based applications in days, not weeks—significantly reducing Stellar application development costs and time-to-market. Instead of doing complex requests using old-school APIs and writing own tooling, developers can use Astrograph to search the ledger using a single GraphQL interface and Astrograph’s own Stellar core data ingester. With Astrograph, it is also possible to work with the Stellar network state in real-time. Astrograph is ready to use, open-source software. We are applying to the SCF to make sure that we’ll have the time to grow and improve Astrograph. Goals Astrograph aims to provide a full-featured GraphQL interface to Stellar network, making the ecosystem more appealing to developers and lowering the entry barrier. Since its first release in April 2019, Astrograph provides access to current and historical state of the ledger, including real-time updates about ongoing changes—all available through a single GraphQL endpoint. The current objectives are: Finish the development of Astrologer — a tool to ingest data from Stellar core node into Elastic Search Switch all queries to use Astrograph's own storage — to reduce the coupling with stellar-core, and remove dependency on Horizon Improve the stability, fix discovered bugs Provide docker images and document deployment scenarios Description : Astrograph is the client-facing GraphQL server for the Stellar ecosystem. It is quite similar to Horizon, except that instead of exposing RESTful API, it provides a single endpoint to explore the Stellar network state and history using powerful GraphQL queries. It also offers reliable WebSocket-based subscriptions, allowing developers to add real-time capabilities to the apps faster and with less effort. Check our blog post to get more details about the project. We run a demo instance of Astrograph so you can explore Stellar through GraphQL right in our playground , we welcome you to start testing your queries. At this point, we primarily want to see how our current schema fits your real-world needs and how it can be improved. Are you building a product based on a Stellar network and think you can benefit from GraphQL’s flexibility and ease of use? Feel free to tell us about your specific needs so we can introduce more queries, connections, filters or subscriptions. Links : Astrograph.io Demo playground Github

Awesome project! I will definitely be playing around with the tick subscription in the next iteration.

Astrograph — Explore the new horizons of Stellar network

tyvdh

Awesome! Great to see some more work going on the GraphQL space for Stellar. A few endpoints I'd like to see added which folks using my solution have used would be showing all accounts holding a particular asset. So essentially trustlines filtered by assetcode. Another would be all operations for a given account. In general just being able to ask account and trustline specific questions is a tremendous improvement over what we get with the Horizon API. The last two popular ones would be the ability query by memo values and account metadata keys and values. We're storing a lot of various things in memos and account data these days so being able to list accounts which match those queries is really nice.

Overall this is a really solid start, especially interested to see where you're able to take this with your own custom ingestion query engine. That's definitely where I've been heading myself as the Horizon database just leaves so much to be desired and the Core has everything obfuscated in XDRs.

Great work so far and good luck!

nebolsin

tyvdh Thank you for the detailed feedback! At this stage our biggest challenge is to understand the actual developer needs and evolve the Astrograph schema accordingly, so we really appreciate your insight and query suggestions. I’m happy to say that Astrograph already supports some of those queries: filtering operations by account or by ledger was there from the launch date, and the PR with asset holders functionality have just landed. I started a wiki page which provides a few examples of currently supported queries.

Filtering transactions by memo is on our roadmap and we’ll get back to it when we finish the internal ingestion engine and no longer depend on Horizon for historical data. As for filtering accounts by metadata, I think that’s a great idea, I created #157 to track the progress. Currently filtering by data entry key looks pretty straightforward, but I’m not quite sure how to handle filtering by value, which is a raw byte array. I guess simple equality condition for byte arrays wouldn’t be very useful. Could you please elaborate a bit on the use case where you need this functionality and what’s the actual data values look like?

Scopuly

Great work! It seems that it solves many of the problems of sampling specific combinations of data from Stellar .. We will follow the development, good luck.

Freitag

This looks like a very good idea to build out! GraphQL APIs are very nice to work with, I would love to integrate your API with my project to generate statistics more easily.

tyvdh

nebolsin Being able to search keys is good but being able to search key values is even better. In my case I use custom data for one of two things.
1) Storing locations for external data. So if I'm recording an account as a holder or issuer of a particular NFT I may record the metadata for that NFT in an external database with a marker in the issuer account's custom data. In this case the key would be unique and the data would be unknown and thus unnecessary to search for.
2) In the other case however I'm starting in an external database and trying to find a particular account by some marker value. So maybe I have marked accounts with ipfs address locations and would like to be able to find the Stellar account which matches one or more of a particular ipfs address which I have stored in custom data. In this case I would search for the value and not the key. Granted I have to search for the serialized value but that's not difficult as Stellar has the method for how they do the serialization so you just have to do that to the term before searching for it.

Hopefully that makes sense. Being able to do both has been very valuable in my own application development and while others have mentioned primarily searching key values and not the data itself both are valuable.

dzham

My primary concern at the moment is being able to filter accounts based on signers and home domains.

dzham

You've probably seen it, but if not: https://thegraph.com/

charlie_wasp

Hi there 👋 Astrograph team here. Recently we introduced a couple of useful queries to Astrograph schema:

filtering accounts by data entry keys and values (as @tyvdh requested):

  accounts(
    data: {
      name: "foo",
      value: "bar"
    },
    first: 5
  ) {   
    nodes {
      id
    }
  }
}

querying assets from issuer’s account node:

    account(id: "GBSTRUSD7IRX73RQZBL3RQUH6KS3O4NYFY3QCALDLZD77XMZOPWAVTUK") {
      assets(first: 5) {
        nodes {
          code
          holdersCount
          unauthorizedHoldersCount
          totalSupply
          authRequired
        }
      }
    }
  }

Playground still available at https://demo.astrograph.io/graph

tyvdh

Awesome! 👏

gzigzigzeo

Hey everyone!

We have finished alfa version of Astrologer: tool that allows to ingest or export historical data from Stellar Core to ElasticSearch.

It stores ledgers, transactions, operations with results and metas in ES and makes you able to search all the data in very flexible way using all the black magic of ElasticSearch engine.

For instance, you could easily build a query which answers the question "What was the most popular operation type three days ago in the whole network?" or "What was the daily average balance of specific account?" or "What was the biggest payment made last week?" or simply find all payments with specific memo value.

Check the repo:

https://github.com/astroband/astrologer, it contains some sample queries in Postman collection.

Astrologer can be used in isolation, as a separate tool, but we are now integrating it into Astrograph. Our first goal is to query operations from ES instead of Horizon (it's almost done), add more search criterias and make some useful aggregation results available via Astrograph GraphQL API.

Ultimate goal is to completely get rid of our dependency on Horizon, and Astrologer integration would be the first step.

We'll be grateful if anyone has any thoughts on queries or use cases which we could implement onset.

tyvdh

Huge news!! Seriously this is amazing. How large is an instance of a fully synced Core?

nebolsin

tyvdh We're still in active development and don't have final numbers for fully synced core. Currently in progress of setting up production Kubernetes cluster on GKE, so should have better estimates for resource and storage requirements in a few weeks.

For development we fit into single DigitalOcean's s-8vcpu-32gb droplet (the specs are 8 vCPU / 32GB RAM / 640GB SSD), which runs everything:

stellar-core in validating mode, which is backfilled without gaps from ledger 20kk to the present moment, so it has all the info for roughly 4 million recent ledgers. It uses local DB and publishes to local history archive (archive is currently 100GB)
postgresql (core DB size is 207GB, biggest tables are txhistory — 155GB, txfeehistory — 36GB, scphistory — 3.5GB).
elasticsearch (currently 155GB total, but it might be bloated with some old dev experiments, etc)
astrologer ingester, which constantly feeds new ledger data from core to ES
astrograph-server which serves our demo GraphQL
astrograph-ingester which feeds realtime subscriptions with new data from core's postgres
prometheus collects metrics
hasura and postgraphile instances targeting the stellar-core DB
some other stuff

gleb

@dzham @nebolsin +1 for abilities to query signers, home domains, inflation stats.

I think support for queries on some of the core data that is not available in horizon, might give developers more motivation to use this kind of tool.

charlie_wasp

gleb Hi! Can you please elaborate, what do you mean by "some of the core data"?

MisterTicot

@nebolsin

Interesting project. As my apps are client-side, I'm an heavy Horizon API consumer. I'm sometimes frustrated by Horizon limitation so I hope something better could be built.

For me to switch, it would require that at least all Horizon data can be fetch from your API. For instance, I could not retrieve the account signers type, which is an important information. (Of course I understand that it is work in progress).

If I can give you an advice, it would be to make sure that Horizon can be emulated on top of your API. From there I would expect devs to start adopting your solution, and even nodes to replace Horizon.

nebolsin

MisterTicot Thank you for the feedback. We're aiming to make Astrograph at least as capable as Horizon, meaning that if a task can be solved with Horizon, it should be also solvable with Astrograph. The lack of signer type is a good catch, and we'll fix it promptly. In the meantime you can easily find out the type from the signer string itself: ed25519 public keys start with 'G', pre-authorized transactions start with 'T' and hashes of the preimages start with 'X'.

Regarding Horizon emulation we're currently considering an option to introduce a small GraphQL service which will use Horizon's REST API as a datasource. That's how Astrograph currently serves historical queries while we're finishing its ElasticSearch ingestion engine. Then we can either include this API into Astrograph main schema via GraphQL schema stitching, or leave it as a separate endpoint.

nebolsin

MisterTicot signer type is part of the schema now: https://github.com/astroband/astrograph/pull/172

deylandra

What the hell is an open-source GraphQL server for Stellar network?

You have to explain it in twenty words or less because I don't like to read large blocks of text, it scares me.

charlie_wasp

Hello, folks! Astrograph team here 🚀 Some quick project updates:

1) We added our own payment paths finding mechanism. We don't use Horizon for it anymore. Here how it looks:

{
  findPaymentPaths(
    sourceAccountID: "GA4DMQ3VSHIVROQ42PJVJOD7X4PYT5BXAWV672CAWOWIADXC3RGZEOMZ",
    destinationAsset: "USD-GBSTRUSD7IRX73RQZBL3RQUH6KS3O4NYFY3QCALDLZD77XMZOPWAVTUK",
    destinationAmount: "0.3"
  ) {
    sourceAsset {
      id
    }
    destinationAsset {
      id
    }
    sourceAmount
    path {
      id
    }
  }
}

There are some differences with Horizon output. Astrograph returns only one best path for each asset in source account's trustlines (plus native lumens). Also, it doesn't search for "detour" if the source account has destination asset in its trustlines.

2) We drop Horizon usage in serving orderbooks too. The schema stays the same

Astrograph still has a playground at https://demo.astrograph.io/graph. And we still would love to hear your feedback 🙂