Rome wasn’t built in a day, and if it was, it would take but a moment to tear down (cheesy, I know).
Well, what I’m getting at is we all know success doesn’t happen overnight. But who am I to talk about success? I’m a 24 year old graduate who is two months into a job I never thought I’d be doing. Ever since I was little, I wanted to be a football player. Only problem is I couldn’t pass, shoot or dribble. But I had the passion. I’m now putting that passion into something I didn’t know I could love. I’m a specialist Data Engineering Recruitment Consultant and I decided to write this article because a lot of the people I speak to in this market either need guidance or have wisdom to give.
So I thought it would be beneficial to reach out to pioneers and experts within my market to heed their advice on how to elevate to the next level.
The first person I spoke to is the Head of Technology at Selligence, Aled Jones, who is responsible for their entire technology function. Selligence are pioneering autonomous predictive intelligence by organizing the world’s information to connect users with the right information, at the right time, for the right reason. Their first go-to-market product, Talent Ticker, can predict the future of your staffing and talent markets and has been globally recognised as the go-to solution for forward-thinking staffing organisations.
Aled’s Top Tips:
Find a company where there is a significant value in the data. There will be more opportunities to provide value.
Gain some familiarity with wider Data Science topics. For example taking online courses, attending meetups or just discussing with peers can help. A better understanding of what the data will be used for will help you to become a better Data Engineer.
Keep abreast of data tooling trends in the industry. Knowing what tool (e.g. database, data warehouse, data lake) to use when is a big part of being a productive Data Engineer.
The second person I spoke to is a Data Engineering Manager at Slice, Amitesh Bhattacharya. Slice have created an innovative tech platform which empowers over 16,000 independent pizzerias with the modern tools that have allowed major pizza chains to dominate until now. Slice is a privately held company that has raised over $120M from former Twitter COO and CEO Adam Bain and Dick Costolo of 01 Advisors, as well as Cross Creek, GGV Capital, KKR, and Primary Venture Partners.
Amitesh’s Top Tips:
Should be able to build new data pipelines that use modern computer languages to create reusable abstractions for data processing, to monitor data pipelines, and to visualize the flow of data.
Experience in Compute engines query in the cloud data without having to move it. This would leverage the separation of data and compute to accelerate queries, enable secure and compliant access.
Help build data products which could be analyses, experiments, reports, and machine learning models/products built on data.
The third person I spoke was the Software Engineering Manager at BAO Systems, Patrick Linton. Patrick is managing an agile team of Data Engineers, Data Scientists, and Project/Product Managers building up the base of the data infrastructure for the PEPFAR (President's Emergency Plan For AIDS Relief) project at the U.S. Department of State's Office of the Global AIDS Coordinator (O/GAC), in collaboration with USAID, HHS, the CDC, the DoD, the DoL, and the Peace Corps.
Patrick’s Top Tips:
Always be willing to learn new things; not just the new cutting-edge technology, but more specifically what you need to know for the job at hand.
Speak up in meetings. If you don’t, you might seem out of touch and your voice will never be hear.
Advocate for yourself. You may be undervalued, so make sure you do the research to know your worth and don’t settle.
And lastly, I had a chat with Paul Boocock – Head of Engineering (Data Engine) at Snowplow Analytics: a behavioural data platform, built to empower data teams to capture and operationalize behavioural data at scale. Paul is responsible for setting the strategic direction for the Snowplow Data Engine teams, who are responsible for the majority of the Open Source Snowplow components (Tracker SDKs, Data Pipeline, Warehouse Loaders, Data Models and Stream Relays).
Paul’s Top Tips:
Never stop learning. The industry is moving at a rapid pace with lots of new tools and preferred ways of building things. So keeping on top of industry trends and understanding what tools are gaining popularity is important.
Don't try to learn everything! As a counter to point 1, there's a lot going on in the Data Engineering space. Use your time wisely in where you invest your effort in learning.
Build your network. Get involved in the community. Make sure you have a LinkedIn presence, and find Slack/Discord communities for Data and Analytics Engineers. Then try and get involved. So many amazing job opportunities get posted in these networks.
Well, there you have it. It has been an absolute pleasure talking to Aled, Amitesh, Patrick and Paul and I can’t thank them enough for giving me their time.
I hope this article has been helpful for anybody either looking to get into Data Engineering or for anyone looking to elevate to that next level.
Smile! It’s almost Easter.😊
Special thanks to:
Aled Jones – Head of Technology – Selligence
Amitesh Bhattacharya – Data Engineering Manager - Slice
Patrick Linton – Software Engineering Manager - BAO Systems
Paul Boocock – Head of Engineering (Data Engine) - Snowplow Analytics