Skip to content

Instantly share code, notes, and snippets.

@jmnavarro

jmnavarro/infrastructure-lead.md

Created March 26, 2021 13:22
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jmnavarro/6ee339f995e60c7da882b98a55a74d6b to your computer and use it in GitHub Desktop.
Save jmnavarro/6ee339f995e60c7da882b98a55a74d6b to your computer and use it in GitHub Desktop.
Download ZIP
Infrastructure Lead position for uDA
Raw
infrastructure-lead.md

Who are we?

At urbanData Analytics, domain experts and engineers team up to provide consultancy services and develop software and data products for the Real Estate industry, one of the few big sectors still on its way to being fully digitalized.

From a technical standpoint, our stack goes all the way from our big data and machine learning engines to our web and Excel (believe it or not, we are really proud of this one!) frontends. You can learn more about how we work by watching this talk (or this other one in Spanish).

Since 2019, uDA has been part of Alantra, the main advisory team for financial institutions, platforms, investors and credit and real estate transactions. With a presence across southern Europe, uDA plans to expand its business and services to the rest of the continent. However, we still keep that spirit of the small, remote-friendly company based out of Madrid that we have been.

What are we looking for?

A Lead Infrastructure Engineer (aka SRE, Systems Engineer, DevOps Engineer, etc.) for our team to work on cloud infrastructure that captures, ingests, processes and delivers huge amounts of data.

Infrastructure must not be a second-class citizen at uDA. We are looking for someone experienced enough to not only do a great engineering work but also help and train other developers on how to do better DevOps. Because you know, DevOps is not a role.

And following that spirit, we want someone willing to contribute in the development side, in order to break silos across dev and ops worlds.

For reference only, this is what our current stack looks like:

  • Google Cloud Platform: Kubernetes, DataFlow, etc.

  • On-premises: we have some in-house servers that need to be managed old-school (Linux sysadmin) for now. We also have plans to migrate them to GCP.

  • Jenkins for deployments and some automations.

  • Python: our lingua franca, used company-wide.

  • PostgreSQL (with PostGIS), MariaDB and Elastic Search: we are mad about them. We are a data centered company and Databases are a key part of our infrastructure.

  • Message brokers: RabbitMQ (and some Google PubSub) because we cannot swallow everything at once!

Are you a good fit?

It is hard to answer a question like this here, but here goes a list of challenges you should find yourself excited about:

  • We are definitely building new and exciting products, but we already have other products out there. They pay our bills and need to be taken care of. Even though the code base and infrastructure is not too old, we have our share of technical debt that needs to be dealt with: understand the reasons behind every workaround, accept the tradeoffs made, evolve iteratively, and find good trade-offs between throwing old stuff away and improving it (starting from scratch is not always an option).

  • Whenever it is actually possible to start from scratch, we try not to be blinded by every trend and buzzword. We need to understand the benefits behind relevant technologies and tools, and choose wisely.

  • Not only do we have big data on our data lake, but also huge tables in Postgres+PostGIS (500M-row tables do not scare us away!). We don't need a pro DBA, but we want you to be open to dive in that space.

  • We are continuously improving our existing data processes and creating new ones. The infrastructure for them can still be defined and there are lots of things to be tried inside and outside Kubernetes.

  • You consider data to be a first-class citizen. Besides "code-deploy", we also have "data-deploy" pipelines because we need to deploy our datasets with the same confidence and control as we do with code.

  • We are transitioning our backend from a big monolith to a more flexible set of independent (macro) services. It is indeed a challenge, but also a great opportunity to improve and optimize.

  • We are investing on taking the way we deploy code and data to the next level by improving the reliability, observability and simplicity of current pipelines and environments. There is the opportunity to analyze and create new CI/CD pipelines and systems.

  • Most of our infrastructure sits on Google Cloud. However, we are far from using it down to its full potential. We need to find cost-effective ways of squeezing the most of it by defining a proper platform governance strategy.

  • All our low-level infra is not properly specified and versioned, so we need to introduce some kind of IaC practice: Terraform, Ansible... you name it.

Great challenges make great fun, do you like it so far?

How will your day-to-day experience be?

  • You will participate in the definition and improvement of a CI/CD system that will work for different kinds of requirements and challenges. Frontend, Backend and Data Engineering teams will all need a robust system as they create new products.

  • You will lay out the foundation of the IaC (Infrastructure as code) for the company.

  • You will help tune our Databases to perform as best as they can.

  • You will design and maintain a secure user and permission policy accross systems.

  • You will improve our monitoring & alerting infrastructure: create container and cluster related alerts, business metrics and better unified logging. In addition, solutions for traces and APM can be investigated.

  • You will work to improve our current and future data acquisition infrastructure together with the Data Engineering team.

  • You will collaborate with the Data Science team in order to understand their needs and setup environments for them: Jupiter notebooks, Machine Learning training & checking pipelines, etc.

  • You will help improve our API development, staging and production environments and associated plumbing.

  • You will motivate and educate team members on DevOps coding standards and best practices.

What can you expect from us?

  • A welcoming bunch of teammates, excited to work with you.

  • Positive + inclusive + respectful working environment.

  • A key position across the most important teams of the company.

  • A focused, eat-our-own-food approach to product development.

  • Competitive salary + bonus.

  • Full remote work in a remote-friendly environment.

  • PluralSight account for learning during your working hours.

  • Personal budget for conferences, events and training.

I'm all in! What should I do?

Send your resume or Linkedin, Git(hub|Lab) profile, or anything you think can help us know you better to jm@urbandataanalytics.com and a brief explanation on why you want to join us!

If you like what you read, but are still not sure if this is the right position for you, please reach out too. Should you have questions or should you want to find out more about other openings we have, we will be more than happy to have a chat with you and talk it over.

Come on! Don't miss the train!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment