For this SBC round I’m digging in deep. My original plan was to rebuild Colorglyph directly into the Stellar mainnet and offer/asset system. However it quickly became clear that for that to actually work well or at all I’d need much better and more powerful access of the Stellar databases. So I fired up a Horizon/Core Docker droplet and started building an API one request at a time. This went on successfully right up until a wild idea hit me. With all the classes and structure from the PostgreSQL database would it be possible to essentially port right to GraphQL and not have to fool with all of these specific endpoints. A little googling and I soon had https://github.com/postgraphql/postgraphql installed and delivering a fully featured GraphQL interface queuing anything and everything in the core and horizon databases. Absolutely amazing. This will change the game for Stellar frontend application development. The rest of my time will be spent building up a wiki of useful boilerplate GraphQL queries as well as writing up a detailed tutorial for getting this set up on your own Stellar nodes.

Feel free to play with the graphql endpoints I have set up. Testnet https://core-test.gly.sh/graphiql https://horizon-test.gly.sh/graphiql Pubnet https://core.gly.sh/graphiql https://horizon.gly.sh/graphiql

Great job! I hope core team bold this out, this is huge good news!

Worth noting I also have test and pub Horizon instances running. https://api-test.gly.sh https://api.gly.sh

@tyvdh I had this same idea today (building a GraphQL layer on top of the ledger). I hadn't thought of using postgraphql, though. If you're willing to open the project up to receive contributions, I'd be happy to be part of it! If not, it'd be great to keep in touch anyway, since I plan to put together a UI that allows the blockchain to be visualised as a graph structure.

codesmithtech Awesome! Definitely planning on opening this up and will be looking for help.

Got a simple Docker image set up for exposing a PostgreSQL database to a GraphQL endpoint via PostGraphile. https://github.com/TinyAnvil/postgraphile-docker https://hub.docker.com/r/tyvdh/postgraphile Next up a docker compose to tie it all together.

And here we are! A fully functional docker-compose set to run the https://github.com/stellar/docker-stellar-core-horizon in concert with https://github.com/TinyAnvil/postgraphile-docker . https://github.com/TinyAnvil/glysh-docker . Now I just need to write a detailed tutorial and maybe record a video walk through.

Medium article laying out how to get everything running on your own servers. https://medium.com/@tyvdh/building-stellar-apps-36303d0e6f45

GraphQL interface for Core and Horizon databases

katopz

For those who want kick start example
1. Open https://core.gly.sh/graphiql
2. Fill this and run

{
  allAccountdata {
    totalCount
    nodes {
      id
    }
  }
}

3. Smile! ?

tyvdh

Recorded a three series of tutorials expanding on that Medium article.

Stellar PostgreSQL to GraphQL: Lesson 1
Stellar PostgreSQL to GraphQL: Lesson 2
Stellar PostgreSQL to GraphQL: Lesson 3

jfuentes

query {
  allHistoryOperations(first: 1, filter: { details: { equalTo: "{ selling_asset_code: \"USD\" }" } }) {
    nodes {
      transactionId
      details
      type
    }
  }
}

tyvdh
I'm trying that query in your https://horizon-test.gly.sh/graphiql and it doesn't work correctly (no data). What I'm missing here?

Thank you.

Edit: actually, filtering by details doesn't seem to work as expected.

dzham

Can we have some examples on how to use this?

tyvdh

dzham Wrote a little reply to the Medium post with some examples.
https://medium.com/@tyvdh/fair-enough-6da46b0734aa

tyvdh

jfuentes I'm using this plugin for the filtering https://github.com/mattbretl/postgraphile-plugin-connection-filter. I have a pending request to see how to filter by a nested JSON key. I wasn't able to get it to work either.

tyvdh

jfuentes Got it. The maintainer of the filter plugin had to make some changes.

query {
  allHistoryOperations(
    first: 1, 
    filter: { 
      details: { 
        contains: { 
          selling_asset_code: "USD" 
        } 
      }
    }
	) {
    nodes {
      transactionId
      details
      type
    }
  }
}

That should do the trick now.

chaynes

tyvdh

This is great stuff, thanks for sharing!

jfuentes

Hey @tyvdh, I was playing a bit with the horizon-test and it went down ?.

Could you please take a look at it?

Thank you!

tyvdh

jfuentes No worries. Back up now. There aren't a whole lot of checks in place atm so if you make a request which returns a MASSIVE amount of data or an overly complex filter the app may crash.

Need to get some checks and auto rebooting in place.

Torkus

Hey Tyler this is absolutely beautiful! Forgive my ignorance about GraphQL but where can we get a list of all entities/fields in order to build some queries? This stuff is amazing!

* facepalm * Top/right there is a Docs link

tyvdh

Torkus Best place is just to explore the "Docs" -> "Query" on the graphiql page itself.

And then too for some inspiration you can read https://medium.com/@tyvdh/fair-enough-6da46b0734aa

Torkus

Thanks, I'm in love!

Hey quick question, can we submit queries using plain http POST? Is there any API endpoint for that? I played with the web UI which is cool but I'd like to make a query once in a while from a program to check for trustlines which is something StellarSDKs lack and it is in urgent need the more tokens get created. Someday when I grow up I'll get my own node and your posts are perfect guides for that but for now I'd be more than happy to hit your service once a day while I get used to server stuff.

* To expand, I saw the links to https://api-test.gly.sh and https://api.gly.sh but do they get graphiql queries in the post body so we get the same responses as the web UI? The ones I tried are just proxies to horizon responses which is not what I want.

tyvdh

Torkus You certainly can! Projects like Apollo https://www.apollographql.com/ make it a lot easier but if you inspect the console on the graphiql page you can see the HTTP requests going out.

Torkus

Another * facepalm * haha so obvious, the web UI has to communicate with the server duh! I'll take a look, thanks!

pdog

This is related to the question posed on stackexchange here: https://stellar.stackexchange.com/questions/729/possible-to-get-a-list-of-account-holders-satisfying-certain-criteria-using-hori/730

If I want a complete list of all native (xlm) accounts that meet a certain criteria, but have the results not just show on my browser but instead get fed into a db, is there a way to take the output from graphiql and get it into a database (there could be 1000's of accounts)? I don't want to overload your setup...

The other option is the one I see mentioned on the above post's answers: setup an instance of stellar core and query the database, and then use that info to build my own database. Is that the only real option here or are there other alternatives?

tyvdh

pdog Certainly! If you're planning on using this in production however I would set the Docker image up on your own servers but so long as you paginate the results and then page through them it should be 100% doable over a period of time as you page through.

MisterTicot

@tyvdh

This is a great project! I totally overlooked it back then.

I'm currently reaching the limits of Horizon with https://equilibre.io. For some portfolios it has keep 20 or more orderbooks up to date, plus the list of offers, and it's badly lagging. It is pretty clear that a GraphQL solution with compound data & subscription could largely improve this situation.

It looks likes what you released so far is a sort of sophisticated demo (no orderbook support for example). Have you any plan implementing in completely?

I have a few questions about GraphQL:

Could it fully replace Horizon?
How does it compare cost-wise with a RESTful solution?
Any way to guard against "abusive" requests?

tyvdh

MisterTicot Hey, there! Appreciate the comments and questions.

To answer the questions in short:

Yes, GraphQL could fully replace all existing Horizon functionality, and it wouldn't be too much work either. Would basically just need to add streaming support and mutation functionality back in.
Cost wouldn't be much different from an operating standpoint, you do have another service to standup and if you want custom functionality you'd have to build that in your self but if you're familiar with postgres or graphql it wouldn't take much time.
Great question and yes, there are several ways abuse could be mitigated. There are several articles and tutorials on the subject. Here's a good one for example. https://blog.apollographql.com/securing-your-graphql-api-from-malicious-queries-16130a324a6b

As far as building these things out myself in gly.sh I'm doubtful. I'd really need to beef up my servers and add a lot of the support for things you've mentioned. While it wouldn't be terribly difficult or expensive, without a consistent Saas type business model it simply wouldn't make sense for me to provide a single solution for everyone. Better for everyone to host, run and customize their own instances at least until the concept of StellarQL matures and we get a better sense of what a single solution might look like as far as features and speed.

tyvdh

MisterTicot Also unless I'm mistaken you can already get the orderbook from the core graphql endpoint with a request something like this.

query allOffers {
  offers: allOffers(
    first: 100
    filter: {
      or: [
        {sellingassetcode: { equalTo: "WSD" }}
        {buyingassetcode: { equalTo: "WSD" }}
      ]
    }
  ) {
    nodes {
      lastmodified
      
      sellingassettype
      sellingassetcode
      sellingissuer
      
      buyingassettype
      buyingassetcode
      buyingissuer
      
    }
  }
}

That would return all offers buying or selling the asset code WSD

MisterTicot

tyvdh

Thank you for the detailled answer. This is absolutely interesting. I think I'll dig in as soon as I can get myself some more free time.

Thank you as well for the offers query. I totally overlooked it.

« Previous Page