Platform Engineering at HubSpot with Tech Lead Gowtam Lal [DevOps Deployed Ep. 01]

I had the chance to chat with Gowtam Lal, a Tech Lead at HubSpot to learn about how platform engineering operates within their organization.

We cover everything from how they use Apache Mesos + Singularity to schedule and manage and schedule 13k+ deployable to how they build systems to manage 50TB of log ingestion every day!

---

Berkeley Mesos paper: people.eecs.berkeley.edu/~alig/papers/mesos.pdf

Hubspot Engineering blog: product.hubspot.com/blog/topic/engineering

---
Timestamps:
00:00 - Intro
01:00 - What is DevOps?
01:48 - What is HubSpot?
03:47 - Team size
04:12 - Platform engineering team
06:11 - Runtime infrastructure
06:44 - Microservice architecture
07:25 - Tooling
08:02 - Apache Mesos + Singularity
10:44 - Monoglot organization (Java)
14:44 - Hiring approach
16:20 - Bootstrapping a new service
20:00 - Java dependency management
22:55 - Monitoring, alerting, and dashboards
25:00 - Logging infrastructure
26:43 - HubSpot scale starting to break community endorsed solutions!
28:00 - Diagnosing the logging challenges
29:00 - Process of rolling out major feature/change
30:44 - Testing strategy
32:22 - Building opinionated solutions
34:38 - Rollout and planning (+ load testing pipeline)
36:17 - Risk assessment across components
37:00 - Feature flag to enable runtime configuration of log emission
38:01 - Major lessons learned (vetted application stack empowers developer productivity)
40:44 - What's next on the roadmap?
43:40 - Go check out the HubSpot engineering blog!

---
Join the Community:
💬 Discord: discord.gg/3XzWctZ
💻 GitHub: github.com/sidpalas/devops-directive
🐥 Twitter: twitter.com/sidpalas
👨‍💼 LinkedIn: www.linkedin.com/in/sid-palas/
🌐 Website: devopsdirective.com/

---

Community size at time of posting:
Subscribers: 11327
Channel Views: