For this SBC round I’m digging in deep. My original plan was to rebuild Colorglyph directly into the Stellar mainnet and offer/asset system. However it quickly became clear that for that to actually work well or at all I’d need much better and more powerful access of the Stellar databases. So I fired up a Horizon/Core Docker droplet and started building an API one request at a time. This went on successfully right up until a wild idea hit me. With all the classes and structure from the PostgreSQL database would it be possible to essentially port right to GraphQL and not have to fool with all of these specific endpoints. A little googling and I soon had https://github.com/postgraphql/postgraphql installed and delivering a fully featured GraphQL interface queuing anything and everything in the core and horizon databases. Absolutely amazing. This will change the game for Stellar frontend application development. The rest of my time will be spent building up a wiki of useful boilerplate GraphQL queries as well as writing up a detailed tutorial for getting this set up on your own Stellar nodes.

Feel free to play with the graphql endpoints I have set up. Testnet https://core-test.gly.sh/graphiql https://horizon-test.gly.sh/graphiql Pubnet https://core.gly.sh/graphiql https://horizon.gly.sh/graphiql

Great job! I hope core team bold this out, this is huge good news!

Worth noting I also have test and pub Horizon instances running. https://api-test.gly.sh https://api.gly.sh

@tyvdh I had this same idea today (building a GraphQL layer on top of the ledger). I hadn't thought of using postgraphql, though. If you're willing to open the project up to receive contributions, I'd be happy to be part of it! If not, it'd be great to keep in touch anyway, since I plan to put together a UI that allows the blockchain to be visualised as a graph structure.

GraphQL interface for Core and Horizon databases

tyvdh

codesmithtech Awesome! Definitely planning on opening this up and will be looking for help.

tyvdh

Got a simple Docker image set up for exposing a PostgreSQL database to a GraphQL endpoint via PostGraphile.

https://github.com/TinyAnvil/postgraphile-docker
https://hub.docker.com/r/tyvdh/postgraphile

Next up a docker compose to tie it all together.

tyvdh

And here we are!

A fully functional docker-compose set to run the https://github.com/stellar/docker-stellar-core-horizon in concert with https://github.com/TinyAnvil/postgraphile-docker.

https://github.com/TinyAnvil/glysh-docker.

Now I just need to write a detailed tutorial and maybe record a video walk through.

tyvdh

Medium article laying out how to get everything running on your own servers.
https://medium.com/@tyvdh/building-stellar-apps-36303d0e6f45

katopz

For those who want kick start example
1. Open https://core.gly.sh/graphiql
2. Fill this and run

{
  allAccountdata {
    totalCount
    nodes {
      id
    }
  }
}

3. Smile! ?

tyvdh

Recorded a three series of tutorials expanding on that Medium article.

Stellar PostgreSQL to GraphQL: Lesson 1
Stellar PostgreSQL to GraphQL: Lesson 2
Stellar PostgreSQL to GraphQL: Lesson 3

jfuentes

query {
  allHistoryOperations(first: 1, filter: { details: { equalTo: "{ selling_asset_code: \"USD\" }" } }) {
    nodes {
      transactionId
      details
      type
    }
  }
}

tyvdh
I'm trying that query in your https://horizon-test.gly.sh/graphiql and it doesn't work correctly (no data). What I'm missing here?

Thank you.

Edit: actually, filtering by details doesn't seem to work as expected.

dzham

Can we have some examples on how to use this?

tyvdh

dzham Wrote a little reply to the Medium post with some examples.
https://medium.com/@tyvdh/fair-enough-6da46b0734aa

tyvdh

jfuentes I'm using this plugin for the filtering https://github.com/mattbretl/postgraphile-plugin-connection-filter. I have a pending request to see how to filter by a nested JSON key. I wasn't able to get it to work either.

tyvdh

jfuentes Got it. The maintainer of the filter plugin had to make some changes.

query {
  allHistoryOperations(
    first: 1, 
    filter: { 
      details: { 
        contains: { 
          selling_asset_code: "USD" 
        } 
      }
    }
	) {
    nodes {
      transactionId
      details
      type
    }
  }
}

That should do the trick now.

chaynes

tyvdh

This is great stuff, thanks for sharing!

jfuentes

Hey @tyvdh, I was playing a bit with the horizon-test and it went down ?.

Could you please take a look at it?

Thank you!

tyvdh

jfuentes No worries. Back up now. There aren't a whole lot of checks in place atm so if you make a request which returns a MASSIVE amount of data or an overly complex filter the app may crash.

Need to get some checks and auto rebooting in place.

Torkus

Hey Tyler this is absolutely beautiful! Forgive my ignorance about GraphQL but where can we get a list of all entities/fields in order to build some queries? This stuff is amazing!

* facepalm * Top/right there is a Docs link

tyvdh

Torkus Best place is just to explore the "Docs" -> "Query" on the graphiql page itself.

And then too for some inspiration you can read https://medium.com/@tyvdh/fair-enough-6da46b0734aa

Torkus

Thanks, I'm in love!

Hey quick question, can we submit queries using plain http POST? Is there any API endpoint for that? I played with the web UI which is cool but I'd like to make a query once in a while from a program to check for trustlines which is something StellarSDKs lack and it is in urgent need the more tokens get created. Someday when I grow up I'll get my own node and your posts are perfect guides for that but for now I'd be more than happy to hit your service once a day while I get used to server stuff.

* To expand, I saw the links to https://api-test.gly.sh and https://api.gly.sh but do they get graphiql queries in the post body so we get the same responses as the web UI? The ones I tried are just proxies to horizon responses which is not what I want.

tyvdh

Torkus You certainly can! Projects like Apollo https://www.apollographql.com/ make it a lot easier but if you inspect the console on the graphiql page you can see the HTTP requests going out.

Torkus

Another * facepalm * haha so obvious, the web UI has to communicate with the server duh! I'll take a look, thanks!

pdog

This is related to the question posed on stackexchange here: https://stellar.stackexchange.com/questions/729/possible-to-get-a-list-of-account-holders-satisfying-certain-criteria-using-hori/730

If I want a complete list of all native (xlm) accounts that meet a certain criteria, but have the results not just show on my browser but instead get fed into a db, is there a way to take the output from graphiql and get it into a database (there could be 1000's of accounts)? I don't want to overload your setup...

The other option is the one I see mentioned on the above post's answers: setup an instance of stellar core and query the database, and then use that info to build my own database. Is that the only real option here or are there other alternatives?

tyvdh

pdog Certainly! If you're planning on using this in production however I would set the Docker image up on your own servers but so long as you paginate the results and then page through them it should be 100% doable over a period of time as you page through.