Skip to content

Instantly share code, notes, and snippets.

@learner-long-life
Last active October 4, 2022 11:54
Show Gist options
  • Save learner-long-life/ee67803f66af7db095062d50b22e3a6f to your computer and use it in GitHub Desktop.
Save learner-long-life/ee67803f66af7db095062d50b22e3a6f to your computer and use it in GitHub Desktop.
How to Run an Ethereum (Geth Light) Node on GCP

How to Run an Ethereum Node in GCP

NOTE: This doc was created in 2018 and is no longer being kept up-to-date. There are some useful updates and Q&A in the comments, and Google has since disallowed cryptocurrency nodes explicitly in their Terms of Service. While I work on an updated how-to-run-a-node tutorial, you are welcome to join our ETH developer chat and share what you'd like to work on.

https://matrix.to/#/#invisible-college/cryptocoin:matrix.org?via=matrix.org&via=t2bot.io

The first step to setting up an Ethereum development environment is running a node. Public nodes run by MyEtherWallet and Augur are great public services. However, you cannot always attach your private keys securely to such a node, and so you are limited to read-only interactions (reading contract variables, checking account balances). Moreover, you cannot always upload contracts reliably due to (Solidity) compiler incompatibilities.

The most reliable option is to run your own node.

You can run a node locally on your laptop or local computer, such as described in the Christmas classic How To Get on Rinkeby Testnet in 10 minutes. However this won't work for people who are behind a firewall that does not permit incoming public TCP/UDP connections on port 30303 (the default), or any new port. Like many home users on their ISP's network. Also, are you expected to just keep your laptop open and running all the time, draining battery power and hogging your coffeeshop's wifi, like a chump? No way!

The answer is to let the world's most too-powerful tech company run your node for you, for free. (Way!)

This write-up will teach you how to run an Ethereum node on Google Cloud Product (GCP).

Google Cloud Product

Create a New Account

You can get a free 1 year trial with $300 worth of credit by signing up with a new email address. Don't worry, you'll be prompted when your free trial expires before your credit card gets charged. If you own your own domain, you can create an infinite supply of email addresses to make bot accounts. (Disclaimer: I neither endorse nor condemn this course of action 👿 )

GCP Signup

Agree and Verify

You'll get a popup asking you if it's okay that they snoop on what restaurants you search for and your current location on your Android phone. (How do you think they got to be the world's most too-powerful company). Do you want to run a free node or not? Pray a hail mary and make peace with your bot's lost privacy, then click I Agree.

Check your email and click the link to verify your account. You'll be taken to your Google Cloud console, but you can get there by going to http://cloud.google.com and clicking Console

Console

The console is your central landing page where you'll start every time you work with GCP. Here you can view all your current services and their usage, as well as create / add new services, access your cloud shell, edit your profile, access your billing settings, etc.

Agree to the Terms of Service (you should probably skim it).

GCP Terms of Service and the Console behind it

Note the banner that says

Sign up for a free trial and you’ll get $300 in credit and 12 months to explore Google Cloud Platform.

Keep an eye on this every time you sign in. You'll see it go down over time. It's pretty hard to make it last one year in my experience.

Virtual Machine

The available services you can run on GCP are grouped into categories. These include databases, message queues, and lots of fancy stuff. We'll start with the most basic service, compute.

The basic unit of compute on GCP is a Virtual Machine. Think of it as a dynamically allocated Linux computer that you can initialize and tear down on demand. We'll use Ubuntu because of its widespread community support, especially on the Ethereum wiki. It also has a large number of available packages so you can download and install your favorite tools with apt.

On the left sidebar menu, under the Compute Engine section you can choose VM Instances, or you can navigate directly to https://console.cloud.google.com/projectselector/compute/instances

You'll see the following screen to create a project (call it something like Ethereum Node) and click Create. GCP Create Project

A project is a collection of instantiated services that are running, with real data, and are consuming both resources like CPU and RAM on Google's machines as well as using up money in your $300 free trial.

You can have multiple projects to work on multiple apps or systems simultaneously. For example after this tutorial is over, you might create another project to run a Bitcoin node as well. We'll just assume we are working with a single Ethereum Node project for now.

If you get lost below, you can always return to your projects by clicking on Console

Enable Billing

You'll be taken to a screen where you will again be tempted to sign up for spam and you sign away more rights, in case you managed to smuggle them past the other Agree buttons that you clicked.

GCP Enable Billing

After you click Agree and continue enter in your customer info, including your credit card info (not shown, lol). This won't be charged after your free trial is over. It's probably just used to verify your identity and safeguard against you using GCP for any illegal purpose.

GCP Enter customer info

Then click Start my free trial, and get a nice welcome.

GCP Welcome to your free trial

Tour the Console

Go ahead and click Tour the console to get a basic primer on how to work this thing. I'll wait. It might not work, i.e. you get a message that says

We can't guide you at this time.

Sad. Anyway, when you're ready, return to the VM Instances dashboard.

Create New VM Instances

Back in your project dashboard, you'll be given an option to create new VMs. There are other options too (import and take the quickstart) but don't do those right now. Import would let you import other VM instances, e.g. from other projects or other GCP users, or snapshots of even this project in the past. Take the quickstart would let you build a sample app.

Create a new VM

In the next screen, you'll get to choose the operating system image. The default is Debian 9, but as discussed above, we'll use Ubuntu. However, if you're experienced and want to run with a more barebones image that you can customize like Debian, etc. you can choose those as well. You're on your own as far as the rest of this tutorial is concerned, however.

New VM instance

Note the helpful hint on the right sidebar that this VM instance will consume $24.67 per month of your free trial.

Click Change and select Ubuntu 16.04 LTS as well.

Choose Ubuntu image

Accept the other defaults.

Then finally click Create.

Log Into VM

You'll be returned to the VM instances dashboard where you can see your instance running. Choose to SSH (log in via secure shell) by choosing Open in browser window.

Browser SSH into VM

The following window will pop up. Congrats! You are now at a normal command-line shell (bash) that you can use to control your VM instance.

This is our goal, a Linux computer for running an Ethereum node that is beyond your ISP's firewall. There is still a firewall of course, but this one is run by Google and is configurable per GCP user and per project.

You have a lot of control over this. Congrats! This concludes the first, and longer, part of the tutorial.

Ethereum Node

At this point, you can stop and use any other tutorial for starting up an Ethereum node (including running a Rinkeby node as described here). Those instructions are agnostic to what computer the node is running on, so just substitute all mentions of "laptop" for "GCP node" instead. They are manual ways of running an Ethereum node.

Install the latest Ethereum software.

sudo apt-get install software-properties-common
sudo add-apt-repository -y ppa:ethereum/ethereum
sudo apt-get update
sudo apt-get install ethereum

Run tmux to get two simultaneous pains.

Start up Geth

Start up a light geth node in one tmux pane.

geth --light

You'll get a lot of output, perhaps after 20 seconds of silence when you're beginning to suspect that you have to open a firewall port after all. But you don't.

Eventually you'll see something like this:

NFO [10-22|23:42:15] Imported new block headers               count=384  elapsed=131.296ms number=4410943 hash=3d2c70…78e1c5 ignored=0
INFO [10-22|23:42:17] Imported new block headers               count=144  elapsed=2.204s    number=4411087 hash=9c0c0e…8c686a ignored=0
INFO [10-22|23:42:17] Generating ethash verification cache     epoch=148 percentage=63 elapsed=3.040s
INFO [10-22|23:42:18] Generated ethash verification cache      epoch=148 elapsed=4.063s
INFO [10-22|23:42:28] Imported new block headers               count=1    elapsed=9.008ms   number=4411088 hash=09509b…2f4f29 ignored=0
INFO [10-22|23:42:34] Imported new block headers               count=1    elapsed=8.219ms   number=4411089 hash=29ab4c…7c926e ignored=0
INFO [10-22|23:42:46] Imported new block headers               count=1    elapsed=8.903ms   number=4411090 hash=5d6bc7…15fcef ignored=0
INFO [10-22|23:42:57] Imported new block headers               count=1    elapsed=8.224ms   number=4411091 hash=62f24e…c15650 ignored=0
INFO [10-22|23:43:17] Imported new block headers               count=1    elapsed=8.382ms   number=4411092 hash=639fbc…ad50b6 ignored=0
INFO [10-22|23:43:41] Imported new block headers               count=1    elapsed=8.847ms   number=4411093 hash=29d8aa…31f833 ignored=0

If you check Ethstats you'll see that you're now in sync with the main blockchain.

@angleton
Copy link

Great article, but it seems to unfortunately stop rather abruptly currenty.

@learner-long-life
Copy link
Author

hi @angleton I'm still working on it. Glad you like it and thanks for the feedback.

@analogrytm
Copy link

How did you manage with a light node on a 10gb disk? I'm about to start building a new node and suspect I would need more space than this.

@zomglings
Copy link

@cryptogoth: Don't you have to set up firewall rules to open up TCP and UDP ingress on 30303 and TCP egress on 30303?

(Or are you connecting only to peers also running on Google's network?)

@sumitpatel93
Copy link

Hi @cryptogoth : I am unable to connect my web3 instance to gcp node.Should I use ext IP or internal IP ? Could you please elaborate the details for firewalls settings also.

@CryptoKiddies
Copy link

When I run eth.syncing it returns false, but I don't see any log output. Is that usual for light clients?

@DuanTranHuy
Copy link

DuanTranHuy commented Jun 8, 2018

@sumitpatel93 i cannot connect too

@AnshikaAgrawal
Copy link

It is saying "flag provided but mot defined --light"

@tpscrpt
Copy link

tpscrpt commented Oct 10, 2018

--syncmode "light"

@1NTheKut
Copy link

1NTheKut commented Dec 6, 2018

Isn't running an ethereum node on a centralized platform like GCP counter-intuitive to decentralization? Or am I mistaken?

@imnik11
Copy link

imnik11 commented Jun 9, 2019

well everything is fine till the geth --light (geth --light is prompting me command not found)

@learner-long-life
Copy link
Author

@imnik11 sounds like you haven't installed geth, or it's not in your path.

@turtlenecksweater good concern. you as an independent user can use your GCP resources however you see fit, and you choosing to run a node on them, and downloading and running node software, is still helping propagate blocks faster, securing the network, and also potentially being an API endpoint for your dapps or those of others.

There is potentially an attack where Google could detect Ethereum nodes running on GCP and perform some coordinated action from the network routing layer (deliberately delaying some blocks in favor of others, or front-running mined blocks) which would be a cool area of future research (e.g. running dark Tor-like nodes, using encrypted Linux volumes and SSL tunnels, encrypting block content, etc.) Google and Amazon so far have declined to give undue attention to blockchains, although that could change when more search traffic and product ranking / commerce occurs on the decentralized web in the future.

@learner-long-life
Copy link
Author

@nkashy1 good question about firewall rules. i don't know the exact answer, but i believe the wire protocol of ethereum does something with UPnP to be able to negotiate port 30303 connections with other nodes, without an explicit firewall. I've been able to sync a node, albeit slowly, through several such firewalls so far.

if you're able to open TCP/UDP ports 30303, i believe you'll connect to peers faster and more reliably.

@learner-long-life
Copy link
Author

@GeeeCoin eth.syncing will be false if you are all caught up on blocks as well, even in light mode (which is just block headers). It's unusual not to see any log output, as you'll still get a new block every 15 seconds.

@learner-long-life
Copy link
Author

@burritofridays i haven't sync'd in awhile, and some reports say a full mainnet node with Parity takes 200 GB these days.
if you have new reports of how much disk space it takes even to run a light node, feel free to mention it here.

@willbeduns
Copy link

I may be sounding kind of like a Tall Poppy here....or maybe even a sniper but why even us GCP use your rooted phone. My phone has 16 cores and more memory than a most towers. I even can render GPU on AMD. So,, this is more for newbies.

@regisd
Copy link

regisd commented Oct 25, 2020

Customer may not use the Services to engage in mining cryptocurrency.
https://cloud.google.com/terms/free-trial

@rifkiamil
Copy link

@cryptogoth augur.net hyperlink is broken in the text

@learner-long-life
Copy link
Author

Thanks for the heads up @regisd I run most of my nodes on AWS these days.

Thanks @rifkiamil I'm not updating this gist anymore since it's against GCP's Terms of Service to run cryptocurrency nodes, and also I believe Augur no longer runs a public node. There's https://infura.io which does ask that you register an account, and many Ethereum dapps rely on them.

@avatar-lavventura
Copy link

How about opening firewall permission for the port?

@learner-long-life
Copy link
Author

How about opening firewall permission for the port?

@avatar-lavventura that is indeed necessary. As discussed above, it's best to run cryptocurrency nodes on a cloud service that explicitly supports them, such as Microsoft's Azure, or own your own personal machines at home.

@avatar-lavventura
Copy link

On your guide you didn't mention to open ports for the google instance. I believe we have to open port 30303 on the google instance setup (or which ever --http.port that is defined for geth ) for other peers to connect into your geth that is running in the google instance.

@learner-long-life
Copy link
Author

Sure, I read back and it is indeed missing the part when opening a firewall. I believe it worked the last time I tried, several years ago, because UPnP was enabled for the GCP firewall, but it may have been disabled since. That's probably for the best, since it is a security hole, and Google's stance on cryptocurrency has remained pretty agnostic or unsupportive.

I'm not a UPnP expert, but here are some reports of geth working together with UPnP routers / firewalls to punch through on port 30303.
ethereum/go-ethereum#890

The --http.port for geth is usually 8545, and is for JSON-RPC requests from a dapp or admin console, whereas port 30303 is for nodes to gossip amongst themselves, using devp2p as a wire protocol. You'll also need to open port 8545 if you need it for your dapp.

@PalmaSolutions
Copy link

I installed it on a self-hosted instance with Debian and I had to follow slightly different instructions, however, after doing so there were no peers and I had to attach geth geth attach and execute this
admin.peers.forEach(function(value){console.log(value.network.remoteAddress+"\t"+value.name)})
as indicated at: https://www.reddit.com/r/ethereum/comments/gb94o1/waiting_for_geth_light_peers_be_like/
worked perfectly afterwards

@jayboro100
Copy link

Create firewall rules that opens up particular port, and update the same tag name in VM network fields/firewall network.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment