Setting up Stellar Core on CentOS 7

bkolobara

Introudction

In this tutorial I will explain how to run a stellar-core node on CentOS 7.2. We will be setting up a full validator, but in most cases you don't need to run a validator. Validation is only necessary if other nodes care about your validation and you want to participate in the SCP. For more information about stellar-core and why you would want to run a node check out this page.

Most steps are the same for setting up a validator or non-validator node, the only difference is in the configuration file. So you can follow this tutorial in both cases.

There are also stellar-core docker images: https://github.com/stellar/docker-stellar-core-horizon
This way you can save a lot of time getting a node up and running by sacrificing some of the flexibility and control.

Requirements

At first I was running a validator on a $5/mo DigitalOcean instance with 512 MB RAM, but I would constantly run into performance issue. Especially, stellar-core would eat up all the RAM and kill the machine when the swap kicked in. I would suggest to have a machine with at least 2 GB of RAM. I didn't run into CPU (average 25%) or bandwidth issue.

General CentOS setup

If you just started a fresh instance of CentOS you should take some steps before the installation of stellar-core to assure the system is secure and up to date.

The first thing I do is to run:

sudo yum upgrade
sudo yum install vim

This will get the repository and all system packages up to date. And of course give me vim so I can easier edit configuration files. Feel free to pick emacs ?

The second task I do, on a freshly installed machine, is to create a local non-root user and forbid root login. You can follow this excellent guide to accomplish this.

I also like to add this line to /etc/ssh/sshd_config:

PasswordAuthentication no

This disables password authentication and you can only use Public/Private Key login on the machine. It's good practice and makes it more secure. No reason to deal with passwords in 2016.

I also strongly encourage you to follow this guide to enable the firewall, set correct time and add a swap file.

I would only leave 2 ports open in this case sshd and the stellar peer port (11625):

sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --permanent --add-port=11625/tcp

Building stellar-core

We will be compiling stellar-core from source and need the developer tools installed:

sudo yum groupinstall 'Development Tools'
sudo yum install postgresql-devel

You can check out all the dependencies required by stellar-core here.

Building gcc

CentOs 7.2 ships with gcc 4.8. Stellar-core requires 4.9 or higher. The best way to solve this is to compile the latest gcc from source following instructions on StackOverflow:
http://stackoverflow.com/questions/36327805/how-to-install-gcc-5-3-with-yum-on-centos-7-2
I would grab the latest version (6.2) instead of 5.3 with:

curl ftp://ftp.gnu.org/pub/gnu/gcc/gcc-6.2.0/gcc-6.2.0.tar.bz2 -O

The compilation process takes some time, go grab a coffee.

You will also need to simlink the freshly compiled libstdc++ library with:

sudo cp /usr/local/lib64/libstdc++.so.6.0.22 /lib64/
sudo rm /lib64/libstdc++.so.6
sudo ln -s /lib64/libstdc++.so.6.0.22 /lib64/libstdc++.so.6

Configuring libpq

At one point during the stellar-core compilation you would run ./configure and it would fail with the following error:
No package 'libpq' found
The problem is that when you install PostgreSQL dev tools on CentOS it doesn't install the required file for pkg-config to find it. To solve this you can create the file by hand.
/usr/lib64/pkgconfig/libpq.pc:

prefix=/usr
libdir=${prefix}/lib64
includedir=${prefix}/include/pgsql
 
Name: LibPQ
Version: 5.5.0
Description: PostgreSQL client library
Requires:
Libs: -L${libdir}/libpq.so -lpq
Cflags: -I${includedir}

Getting stellar-core

Just grab it from git and check out the latest release tag:

git clone https://github.com/stellar/stellar-core.git
git checkout v0.5.0 # Check out the latest stable version
git submodule init
git submodule update

Compiling & installing stellar-core

The following steps will compile and install stellar-core:

./autogen.sh
./configure
make -j 4
make check # Runs the tests
sudo make install

Configuring CentOs for stellar-core

We will run stellar-core under the stellar user:

sudo useradd stellar

Note that we don't set a password for this user as we will never need it. If we want to become the user for configuration purposes we can always do 'sudo su - stellar'.

Installing PostgreSQL

For the production database you probably don't want to use SQLite. It's a good idea to install PostgreSQL. Here are instructions how to install it on CentOS:
https://www.digitalocean.com/community/tutorials/how-to-install-and-use-postgresql-on-centos-7

You will need to create a role stellar and a database stellar:

sudo -i -u postgres
createuser --interactive # role=stellar, everything else=no
createdb stellar

Again, we don't need to set a password for the user, because stellar-core will run under the user stellar and as we have the same roll and database it automatically has privileges to access it.

Stellar folders

Stellar requires a place on disk to save logs and the full ledger:

sudo mkdir /var/log/stellar
sudo chown stellar /var/log/stellar
 
sudo mkdir -p /var/stellar/buckets
sudo chown stellar /var/stellar
sudo chown stellar /var/stellar/buckets

Configuring stellar-core

This is the most important step! You can find more information about all the configuration parameters here.

This is an example validator configuration file:
https://github.com/stellar/docs/blob/master/other/stellar-core-validator-example.cfg

I usually start from the example file and adapt it to my needs:

sudo mkdir /etc/stellar
sudo cp stellar-core/docs/stellar-core_example.cfg /etc/stellar/stellar.cfg
sudo chown stellar /etc/stellar/stellar.cfg
sudo chmod 600 /etc/stellar/stellar.cfg
sudo vim /etc/stellar/stellar.cfg

An example of some of my configuration parameters:

LOG_FILE_PATH="/var/log/stellar/stellar-core.log"
BUCKET_DIR_PATH="/var/stellar/buckets"
DATABASE="postgresql://dbname=stellar user=stellar"
 
NODE_NAMES=[
"GAOO3LWBC4XF6VWRP5ESJ6IBHAISVJMSBTALHOQM2EZG7Q477UWA6L7U  eno",
"GAXP5DW4CVCW2BJNPFGTWCEGZTJKTNWFQQBE5SCWNJIJ54BOHR3WQC3W  moni",
"GBFZFQRGOPQC5OEAWO76NOY6LBRLUNH4I5QYPUYAK53QSQWVTQ2D4FT5  dzham",
"GDXWQCSKVYAJSUGR2HBYVFVR7NA7YWYSYK3XYKKFO553OQGOHAUP2PX2  jianing",
"GCJCSMSPIWKKPR7WEPIQG63PDF7JGGEENRC33OKVBSPUDIRL6ZZ5M7OO  tempo.eu.com",
"GCCW4H2DKAC7YYW62H3ZBDRRE5KXRLYLI4T5QOSO6EAMUOE37ICSKKRJ  sparrow_tw",
"GD5DJQDDBKGAYNEAXU562HYGOOSYAEOO6AS53PZXBOZGCP5M2OPGMZV3  fuxi.lab",
"GBGGNBZVYNMVLCWNQRO7ASU6XX2MRPITAGLASRWOWLB4ZIIPHMGNMC4I  huang.lab",
"GDPJ4DPPFEIP2YTSQNOKT7NMLPKU2FFVOEIJMG36RCMBWBUR4GTXLL57  nezha.lab",
"GCDLFPQ76D6YUSCUECLKI3AFEVXFWVRY2RZH2YQNYII35FDECWUGV24T  SnT.Lux",
"GBAR4OY6T6M4P344IF5II5DNWHVUJU7OLQPSMG2FWVJAFF642BX5E3GB  telindus",
# non validating
"GCGB2S2KGYARPVIA37HYZXVRM2YZUEXA6S33ZU5BUDC6THSB62LZSTYH  sdf_watcher1",
"GCM6QMP3DLRPTAZW2UZPCPX2LF3SXWXKPMP3GKFZBDSF3QZGV2G5QSTK  sdf_watcher2",
"GABMKJM6I25XI4K7U6XWMULOUQIQ27BCTMLS6BYYSOWKTBUXVRJSXHYQ  sdf_watcher3"
]
 
PREFERRED_PEERS=[]
 
PREFERRED_PEER_KEYS=[
"$sdf_watcher1",
"$sdf_watcher2",
"$sdf_watcher3",
"$dzham",
"$SnT.Lux",
"$tempo.eu.com"
]
 
KNOWN_PEERS=[
"core-live-a.stellar.org:11625",
"core-live-b.stellar.org:11625",
"core-live-c.stellar.org:11625",
"confucius.strllar.org",
"stellar1.bitventure.co",
"stellar.256kw.com"
]
 
NODE_SEED="SXXXXXX....XXXX galactictalk.org"
 
NODE_IS_VALIDATOR=true
 
# History local commented out
# [HISTORY.local]
# get="cp /var/stellar/history/vs/{0} {1}"
# put="cp {0} /var/stellar/history/vs/{1}"
# mkdir="mkdir -p /var/stellar/history/vs/{0}"
 
# Stellar.org history store
[HISTORY.sdf1]
get="curl -sf http://history.stellar.org/prd/core-live/core_live_001/{0} -o {1}"
 
[HISTORY.sdf2]
get="curl -sf http://history.stellar.org/prd/core-live/core_live_002/{0} -o {1}"
 
[HISTORY.sdf3]
get="curl -sf http://history.stellar.org/prd/core-live/core_live_003/{0} -o {1}"
 
[QUORUM_SET]
VALIDATORS=[
"$moni",
"$eno",
"$tempo.eu.com",
"$dzham",
"$sparrow_tw",
"$SnT.Lux"
]

# This field does not exist
# MAINTENANCE_ON_STARTUP=true

Notice how I commented out the MAINTENANCE_ON_STARTUP parameter. I do this because I was getting an error that this parameter does not exist when starting stellar-core:

<startup> [default FATAL] Got an exception: Unknown configuration entry: 'MAINTENANCE_ON_STARTUP' [main.cpp:537]

As a validator, defining the QUORUM_SET is going to be one of the most important things you do. Take your time to read more about it and understand it.

Create the stellar-core tables in PostgreSQL

Switch to the stellar user and create the database tables:

sudo su - stellar
stellar-core --conf /etc/stellar/stellar.cfg --newdb

Add stellar-core to systemd

If the previous step went well you want to add stellar-core to systemd as a service, so it automatically restarts on crashes and gets started when the system starts.
To do this create the file /etc/systemd/system/stellar-core.service with the following content:

[Unit]
Description=Stellar Core
After=postgresql.service
 
[Service]
ExecStart=/usr/local/bin/stellar-core --conf /etc/stellar/stellar.cfg
User=stellar
Group=stellar
WorkingDirectory=/home/stellar
Restart=on-failure
 
[Install]
WantedBy=default.target

Notice how stellar-core is started after PostgreSQL so it has the database available.

Now you can start your stellar-core node and enable automatic startup:

sudo systemctl start stellar-core
sudo systemctl enable stellar-core

Inspecting the state of your node

Congratulations! Your node should be up and running. Now you want to keep one eye on the logs and the health of the quorum so your node stays up and happy.

To check what the stellar-core service is writing to stdout and stderr run:

sudo journalctl -u stellar

For the rest of the commands you need to be the stellar user:

sudo su - stellar

To get general information about your node you can run:

stellar-core --conf /etc/stellar/stellar.cfg --c 'info'

Or to check the quorum status:

stellar-core --conf /etc/stellar/stellar.cfg --c 'quorum'

An explanation of all this fields you can find here.

Hope this tutorial was useful. If you get stuck on some steps or require additional clarification feel free to ask here.

antb123

A couple bits of feed back:
- Network latency is actually the weak point in stellar-core performance so you need a decent hosting partner with a good network (>1 GB connection preferable)
- If you use Debian/Ubuntu there are pre-compiled versions of stellar-core available directly from Stellar
- If you want to add your node to https://stellar.network contact the admin there
- Moving SSH off port 22 to a port under 1024 is a good idea to protect against 0day attacks

zcc

antb123

If you use Debian/Ubuntu there are pre-compiled versions of stellar-core available directly from Stellar

Do you have a link to these?

jed

Also note if bandwidth is an issue reduce the number of peers your server connects to

Eno

as stellar-core write its own log,

giving

[Service]
StandardOutput=null

will skip duplicate messages in syslog file(ubuntu 15.04).

Nusquam

Hi Everyone,

I'm on a Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz server with Centos 7 distribution and gcc-7.1.0.
When I launch ./make command, I have a lot of errors about « BigDivideTests.cpp » file and the compilation crash.
Maybe someone will have an idea...

depbase=`echo util/BigDivideTests.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
g++ -DHAVE_CONFIG_H -I. -I..  -DASIO_SEPARATE_COMPILATION=1 -DSQLITE_OMIT_LOAD_EXTENSION=1 -I".." -I"../src" -I"../src" -I/home/constellat/stellar-core/stellar-core/lib/libsodium/src/libsodium/include -I../lib/libsodium/src/libsodium/include -I../lib/libsodium/src/libsodium/include/sodium -I/home/constellat/stellar-core/stellar-core/lib/xdrpp -I/home/constellat/stellar-core/stellar-core/lib/xdrpp -I../lib/libmedida/src -I../lib/soci/src/core -I../lib/soci/src/backends/sqlite3 -I../lib/soci/src/backends/postgresql -I../lib/sqlite -I"../lib" -I"../lib/autocheck/include" -I"../lib/cereal/include" -I"../lib/asio/include" -I"../lib/util" -I"../lib/soci/src/core" -DUSE_POSTGRES=1 -I/usr/include/pgsql     -g -O2  -pthread -Wall -Wno-unused-command-line-argument -Wno-unused-local-typedef -Wno-unknown-warning-option -Werror=unused-result -MT util/BigDivideTests.o -MD -MP -MF $depbase.Tpo -c -o util/BigDivideTests.o util/BigDivideTests.cpp &&\
mv -f $depbase.Tpo $depbase.Po
util/BigDivideTests.cpp:11:43: error: ‘function’ in namespace ‘std’ does not name a template type
 template <typename T> using Verify = std::function<void(T, T, T, T, T)>;
                                           ^~~~~~~~
util/BigDivideTests.cpp:16:20: error: ‘function’ in namespace ‘std’ does not name a template type
     using P = std::function<bool(uint128_t, uint128_t, uint128_t)>;
                    ^~~~~~~~
util/BigDivideTests.cpp:17:20: error: ‘function’ in namespace ‘std’ does not name a template type
     using R = std::function<uint128_t(uint128_t, uint128_t, uint128_t)>;
                    ^~~~~~~~
util/BigDivideTests.cpp:19:53: error: ‘Verify’ has not been declared
     explicit BigDivideTester(std::vector<T> values, Verify<T> const& verify)
                                                     ^~~~~~
util/BigDivideTests.cpp:19:59: error: expected ‘,’ or ‘...’ before ‘<’ token
     explicit BigDivideTester(std::vector<T> values, Verify<T> const& verify)
                                                           ^
util/BigDivideTests.cpp:25:10: error: ‘P’ has not been declared
     test(P const& p, R const& downResult, R const& upResult)
          ^
util/BigDivideTests.cpp:25:22: error: ‘R’ has not been declared
     test(P const& p, R const& downResult, R const& upResult)
                      ^
util/BigDivideTests.cpp:25:43: error: ‘R’ has not been declared
     test(P const& p, R const& downResult, R const& upResult)
                                           ^
util/BigDivideTests.cpp:40:5: error: ‘Verify’ does not name a type; did you mean ‘erfl’?
     Verify<T> mVerify;
     ^~~~~~
     erfl
util/BigDivideTests.cpp: In constructor ‘BigDivideTester<T>::BigDivideTester(std::vector<T>, int)’:
util/BigDivideTests.cpp:20:39: error: class ‘BigDivideTester<T>’ does not have any field named ‘mVerify’
         : mValues{std::move(values)}, mVerify{verify}
                                       ^~~~~~~
util/BigDivideTests.cpp:20:47: error: ‘verify’ was not declared in this scope
         : mValues{std::move(values)}, mVerify{verify}
                                               ^~~~~~
util/BigDivideTests.cpp:20:47: note: suggested alternative: ‘erfl’
         : mValues{std::move(values)}, mVerify{verify}
                                               ^~~~~~
                                               erfl
util/BigDivideTests.cpp: In member function ‘void BigDivideTester<T>::test(const int&, const int&, const int&)’:
util/BigDivideTests.cpp:30:77: error: expression cannot be used as a function
                     if (c != 0 && p(uint128_t{a}, uint128_t{b}, uint128_t{c}))
                                                                             ^
util/BigDivideTests.cpp:33:77: error: expression cannot be used as a function
                                     uint128_t{a}, uint128_t{b}, uint128_t{c})),
                                                                             ^
util/BigDivideTests.cpp:35:77: error: expression cannot be used as a function
                                     uint128_t{a}, uint128_t{b}, uint128_t{c})));
                                                                             ^
util/BigDivideTests.cpp: In function ‘void ____C_A_T_C_H____T_E_S_T____43()’:
util/BigDivideTests.cpp:131:76: error: no matching function for call to ‘BigDivideTester<long int>::BigDivideTester(<brace-enclosed initializer list>)’
     auto signedTester = BigDivideTester<int64_t>{signedValues, verifySigned};
                                                                            ^
util/BigDivideTests.cpp:19:14: note: candidate: BigDivideTester<T>::BigDivideTester(std::vector<T>, int) [with T = long int]
     explicit BigDivideTester(std::vector<T> values, Verify<T> const& verify)
              ^~~~~~~~~~~~~~~
util/BigDivideTests.cpp:19:14: note:   no known conversion for argument 2 from ‘____C_A_T_C_H____T_E_S_T____43()::<lambda(int64_t, int64_t, int64_t, int64_t, int64_t)>’ to ‘int’
util/BigDivideTests.cpp:13:29: note: candidate: BigDivideTester<long int>::BigDivideTester(const BigDivideTester<long int>&)
 template <typename T> class BigDivideTester
                             ^~~~~~~~~~~~~~~
util/BigDivideTests.cpp:13:29: note:   candidate expects 1 argument, 2 provided
util/BigDivideTests.cpp:13:29: note: candidate: BigDivideTester<long int>::BigDivideTester(BigDivideTester<long int>&&)
util/BigDivideTests.cpp:13:29: note:   candidate expects 1 argument, 2 provided
util/BigDivideTests.cpp:133:65: error: no matching function for call to ‘BigDivideTester<long unsigned int>::BigDivideTester(<brace-enclosed initializer list>)’
         BigDivideTester<uint64_t>{unsignedValues, verifyUnsigned};
                                                                 ^
util/BigDivideTests.cpp:19:14: note: candidate: BigDivideTester<T>::BigDivideTester(std::vector<T>, int) [with T = long unsigned int]
     explicit BigDivideTester(std::vector<T> values, Verify<T> const& verify)
              ^~~~~~~~~~~~~~~
util/BigDivideTests.cpp:19:14: note:   no known conversion for argument 2 from ‘____C_A_T_C_H____T_E_S_T____43()::<lambda(uint64_t, uint64_t, uint64_t, uint64_t, uint64_t)>’ to ‘int’
util/BigDivideTests.cpp:13:29: note: candidate: BigDivideTester<long unsigned int>::BigDivideTester(const BigDivideTester<long unsigned int>&)
 template <typename T> class BigDivideTester
                             ^~~~~~~~~~~~~~~
util/BigDivideTests.cpp:13:29: note:   candidate expects 1 argument, 2 provided
util/BigDivideTests.cpp:13:29: note: candidate: BigDivideTester<long unsigned int>::BigDivideTester(BigDivideTester<long unsigned int>&&)
util/BigDivideTests.cpp:13:29: note:   candidate expects 1 argument, 2 provided
At global scope:
cc1plus: warning: unrecognized command line option ‘-Wno-unknown-warning-option’
cc1plus: warning: unrecognized command line option ‘-Wno-unused-local-typedef’
cc1plus: warning: unrecognized command line option ‘-Wno-unused-command-line-argument’
make[3]: *** [util/BigDivideTests.o] Error 1
make[3]: Leaving directory `/home/constellat/stellar-core/stellar-core/src'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/home/constellat/stellar-core/stellar-core/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/constellat/stellar-core/stellar-core'
make: *** [all] Error 2

bkolobara

Nusquam I really don't know what this error could be related to. If this is only a test file maybe it's possible to compile it without the tests. Are you compiling from master or a specific version?

Nowadays I compile stellar-core on Alpine Linux to a minimalistic docker image. I was receiving some errors related to musl libc that I needed to manually patch, but yours look more like a not-compatible-compiler issue.

gav

I am seeing the same problem when I am trying to compile as well. Any fix for this?

bkolobara

gav Maybe opening a ticket about it on the official issue tracker will get more developer eyes on the problem: https://github.com/stellar/stellar-core/issues

gav

We were able to compile from source after playing around with it for several days. Only had CentOs, so had to stick with that.

sgehrman

To get systemd working on Ubuntu, I had to use:

[Install]
WantedBy=multi-user.target