Responsibilities
- As an individual contributor, design and implement processes, tools, automation, and libraries that service teams can use to improve the reliability of the services they own. For instance, adding a new long-awaited feature in our circuit breaker library.
- Introduce chaos engineering to Canva and conduct experiments to identify possible scenarios in which cascading failure might occur and to verify the reliability measures we introduce to prove this works as expected. E.g. discovering what will happen when this newly introduced service goes down? Does the fallback for this rare failure actually work?
- Work with product engineering teams to ensure reliability best practices and tools are rolled out in every service across the whole organization. It’s not enough to create a new throttling library, we want to make sure it’s successfully used in every service.
- Foster a culture within the Engineering org that puts reliability first and establish processes and policies that drive reliability within product engineering teams. This includes things like SLAs, error budgets, on-call response, incident resolution, observability best practices.
- Deep investigation into production incidents followed up by applying the learning to code.
- Researching, developing and justifying the best choices in the form of design docs for tools and processes that will shape the future of reliability at Canva.
- Propose new approaches and solutions to ensure we future-proof Canva’s distributed cloud infrastructure as we scale.
- Participating in design meetings, hiring interviews, and code reviews.
Required Skills & Experience
- Five-plus (5+) years of commercial experience working with developing complex, distributed web applications.
- Experience working with a mainstream programming language. However, our services and libraries are primarily written in Java 13, so Java is nice to have.
- Solid understanding of resiliency techniques and patterns – load balancing, throttling, back pressure, circuit breaking, etc;
- Disciplined coding practices, experience with code reviews and pull requests and a creative and conceptual problem-solving approach.
- Strong communication and team collaboration skills, both written and verbal. As a reliability engineer, you will need to share the knowledge, communicate and coordinate changes across multiple service teams.
Nice to have; Not required!
- Experience working with microservice architectures in large distributed cloud environments (ideally AWS). We’re hosted on AWS and leverage the tools they provide as much as possible
- Experience with RPC Frameworks, Finagle, Thrift or gRPC will be a huge plus, but not required; Understanding how services communicate with each other is crucial to find out where a failure can occur.
- Knowledge of networking protocols such as TCP, HTTP/2, WebSockets, etc. would be a big plus; The life of a request doesn’t start inside the backend web server, but rather in the browser of a user.
- Previous experience of working as a reliability/chaos engineer and/or strong knowledge of Google SRE
Benefits
- Competitive salary, plus equity options
- Flexible daily working hours and open to remote, we value work-life balance
- In-house chefs that cook delicious breakfast and lunch for us each day
- Onsite Gym; Yoga Benefits
- Generous parental (including secondary) leave policy
- Pet-friendly offices
- Sponsored social clubs and team events
- Relocation budget for interstate or overseas individuals that legally qualify for visa sponsorship