Site reliability engineering
 | This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages)
|
Site reliability engineering (SRE) is a set of principles and practices that applies aspects of software engineering to IT infrastructure and operations.[1] SRE creates scalable IT systems. SRE is similar to DevOps, a set of software and development practices.[2][3][4]
History
[edit]The field of SRE originated at Google with Ben Treynor Sloss,[5][6] who founded a site reliability team in 2003.[7] The concept spread into the broader software development industry, and other companies subsequently began to employ site reliability engineers.[8] By 2016, Google employed more than 1,000 site reliability engineers.[9] Dedicated SRE teams are common at larger web companies. DevOps team sometimes serve the dual purpose of SRE in midsize and smaller companies.[8] Organizations that have adopted the concept include Airbnb, Dropbox, IBM,[10] LinkedIn,[11] Netflix,[9] and Wikimedia.[12] According to a 2021 report by the DevOps Institute, 22% of respondents in a survey of 2,000 worldwide IT professionals had adopted the SRE model compared to 15% percent the previous year.[13][14]
Definition
[edit]Site reliability engineering as a job role may be performed by individual contributors or organized in teams, responsible for a combination of the following within a broader engineering organization: System availability, latency, performance, efficiency, change management, monitoring, emergency response, and capacity planning.[15] Site reliability engineers often have backgrounds in software engineering, system engineering, or system administration.[16] Focuses of SRE include automation, system design, and improvements to system resilience.[16]
The set of principles and practices in site reliability engineering can be performed by anyone. Though other team members should carry out good practices, as in security engineering, a company may eventually hire specialists and engineers for the job.[citation needed]
SRE is considered a specific implementation of DevOps;[17] SRE focuses specifically on building reliable systems, whereas DevOps covers a broad scope.[2][3][4] Despite having different focuses, some companies have re-branded their operations teams to SRE teams with little meaningful change.[8]
Principles and practices
[edit]There have been multiple attempts to define a canonical list of site reliability engineering principles, but while consensus is lacking, the following characteristics are usually included in most definitions:[1][18]
- Automation of repetitive tasks for cost-effectiveness
- Constrain the pursuit of reliability to the pre-defined reliability goals. Defining these reliability goals is one of the SRE practices (see list of practices below).
- Design of systems with a bias toward the reduction of risks to availability, latency, and efficiency.
- Observability i.e. the ability to ask arbitrary questions about a system without having to know ahead of time what to ask.[19]
The site reliability engineering practices also vary widely, but the list below is relatively commonly seen as at least partially implemented:
- Toil management as the implementation of the first principle outlined above.
- Defining and measuring reliability goals—SLIs, SLOs, and error budgets.
- Non-Abstract Large Scale Systems Design (NALSD) with a focus on reliability.
- Designing for and implementing observability.
- Defining, testing, and running an incident management process.
- Capacity planning.
- Change and release management, including CI/CD.
- Chaos engineering.
Implementations
[edit]SRE teams collaborate with other departments within organizations to implement principles effectively. Below is an overview of common practices:[20]
Kitchen Sink, a.k.a. “Everything SRE”
[edit]Kitchen Sink refers to the expansive and often unbounded scope of services and workflows that SRE teams oversee. Unlike traditional roles with clearly defined boundaries, SREs are tasked with various responsibilities, including system performance optimization, incident management, and automation. This holistic approach allows SREs to address multiple challenges, ensuring that systems run efficiently and evolve in response to changing demands and complexities. By embracing this comprehensive perspective, SRE teams are meant to promote continuous improvement and resilience.
Infrastructure
[edit]Infrastructure SRE teams focus on maintaining and improving the reliability of systems that support other teams’ workflows. While they sometimes collaborate with platform engineering teams, their primary responsibility is ensuring up-time, performance, and efficiency. Platform teams, on the other hand, primarily develop the software and systems used across the organization. While reliability is a goal for both, platform teams prioritize creating and maintaining the tools and services used by internal stakeholders, whereas Infrastructure SRE teams are tasked with ensuring those systems run smoothly and meet reliability standards.
Tools
[edit]Teams utilize a variety of tools to measure, maintain, and enhance system reliability. These tools play a role in monitoring performance, identifying issues, and facilitating proactive maintenance. For instance, Nagios Core is widely used for system monitoring and alerting, while Prometheus (software) is popular for collecting and querying metrics in cloud-native environments. Leveraging these tools, SRE teams can ensure optimal performance and quickly respond to potential reliability challenges.
Product or application
[edit]SRE teams dedicated to specific products or applications are common in large organizations. These teams are responsible for ensuring the reliability, scalability, and performance of key services. In larger companies, it's typical to have multiple SRE teams, each focusing on different products or applications, ensuring that each area receives specialized attention to meet performance and availability targets
Embedded
[edit]In an embedded model, individual SREs or small SRE pairs are integrated within software engineering teams. These SREs work closely with developers, applying core SRE principles, such as automation, monitoring, and incident response—directly to the software development lifecycle. This approach helps improve reliability, performance and collaboration between SREs and developers.
Consulting
[edit]Consulting SRE teams specialize in advising organizations on the implementation of SRE principles and practices. Typically composed of seasoned SREs with extensive experience across various implementations, these teams provide valuable insights and guidance tailored to specific organizational needs. When working directly with clients, these SREs are often referred to as 'Customer Reliability Engineers.'
In large organizations that have adopted SRE, a hybrid model is common. This model includes various implementations, such as multiple Product/Application SRE teams dedicated to addressing the unique reliability needs of different products. An Infrastructure SRE team may collaborate with a Platform engineering group to achieve shared reliability goals for a unified platform that supports all products and applications
Industry
[edit]Since 2014, the USENIX organization has hosted the annual SREcon conference, bringing together site reliability engineers from various industries. This conference is a platform for professionals to share knowledge, explore best practices, and discuss trends in site reliability engineering.[21]
See also
[edit]References
[edit]- ^ a b "Evaluating where your team lies on the SRE spectrum". Google Cloud Blog. Retrieved 2021-06-26.
- ^ a b Beyer, Betsy; Jones, Chris; Petoff, Jennifer; Murphy, Niall, eds. (2016). Site Reliability Engineering: How Google Runs Production Systems. Sebastopol, CA: O'Reilly Media. ISBN 978-1-4919-5118-7. OCLC 945577030.
- ^ a b Vargo, Seth; Fong-Jones, Liz (March 1, 2018). What's the Difference Between DevOps and SRE? (class SRE implements DevOps) (Video). Google.
- ^ a b "What is SRE? - SRE Explained - AWS". Amazon Web Services, Inc. Retrieved 2022-11-05.
- ^ Hill, Patrick. "Love DevOps? Wait until you meet SRE". Atlassian. Retrieved June 17, 2021.
- ^ "What is SRE?". Red Hat. Retrieved June 17, 2021.
- ^ Treynor, Ben (2014). "Keys to SRE". USENIX SREcon14. Retrieved June 17, 2021.
- ^ a b c Gossett, Stephen (June 1, 2020). "What Is a Site Reliability Engineer? What Does an SRE Do?". Built In. Retrieved June 17, 2021.
- ^ a b Fischer, Donald (March 2, 2016). "Are site reliability engineers the next data scientists?". TechCrunch. Retrieved June 17, 2021.
- ^ "Site Reliability Engineering". IBM Cloud Education. IBM. November 12, 2020. Retrieved June 21, 2021.
- ^ "Site Reliability Engineering (SRE)". engineering.linkedin.com. Retrieved March 12, 2024.
- ^ "SRE - Wikitech". wikitech.wikimedia.org. Retrieved 2021-10-17.
- ^ Oehrlich, Eveline; Groll, Jayne; Garbani, Jean-Pierre (2021). Upskilling 2021 Enterprise DevOps SkillsReport (PDF) (Report). DevOps Institute. Retrieved June 17, 2021.
- ^ Oehrlich, Eveline (May 4, 2021). "What it takes to be a site reliability engineer". TechBeacon. Micro Focus. Retrieved June 17, 2021.
- ^ Treynor, Ben. "In Conversation" (Interview). Interviewed by Niall Murphy. Google Site Reliability Engineering.
- ^ a b Jones, Chris; Underwood, Todd; Nukala, Shylaja (June 2015). "Hiring Site Reliability Engineers" (PDF). ;login:. Vol. 40, no. 3. pp. 35–39. Retrieved June 17, 2021.
- ^ Dave Harrison (9 Oct 2018). "Interview with Betsy Beyer, Stephen Thorne of Google". Retrieved 24 July 2024.
- ^ "The 7 SRE Principles [And How to Put Them Into Practice]". www.blameless.com. Retrieved 2021-06-26.
- ^ "Learn about observability | Honeycomb". docs.honeycomb.io. Retrieved 2021-06-26.
- ^ "SRE at Google: How to structure your SRE team". Google Cloud Blog. Retrieved 2021-06-26.
- ^ "Usenix SREcon". USENIX. 2021. Retrieved June 17, 2021.
Further reading
[edit]- Limoncelli, Tom; Chalup, Strata R.; Hogan, Christina J. (September 2014). The Practice of Cloud System Administration: DevOps and SRE Practices for Web Services. Vol. 2. Upper Saddle River, NJ: Addison-Wesley. ISBN 978-0133478549. OCLC 891786231.
- Beyer, Betsy; Jones, Chris; Petoff, Jennifer; Murphy, Niall Richard, eds. (2016). Site Reliability Engineering: How Google Runs Production Systems. O'Reilly. ISBN 978-1491929124.
- Blank-Edelman, David N., ed. (2018). Seeking SRE: Conversations About Running Production Systems at Scale (1 ed.). Sebastopol, CA: O'Reilly. ISBN 978-1491978863. OCLC 1052565720.
- Beyer, Betsy; Murphy, Niall; Kawahara, Kent; Rensin, David; Thorne, Stephen (2018). The Site Reliability Workbook: Practical Ways to Implement SRE. O'Reilly. ISBN 978-1492029502.
- Welch, Nat (2018). Real-World SRE: The Survival Guide for Responding to a System Outage and Maximizing Uptime. Packt. ISBN 978-1788628884.
- Adkins, Heather; Beyer, Betsy; Blankinship, Paul; Lewandowski, Piotr; Oprea, Ana; Stubblefield, Adam (2020). Building Secure and Reliable Systems: Best Practices for Designing, Implementing, and Maintaining Systems. O'Reilly. ISBN 978-1-4920-8312-2. OCLC 1129470292.
- Rosenthal, Jones, Casey, Nora (2020). Chaos Engineering: System Resiliency in Practice. O'Reilly. ISBN 978-1492043867.
{{cite book}}: CS1 maint: multiple names: authors list (link)
External links
[edit]- Awesome Site Reliability Engineering resources list
- How they SRE resources list
- SRE Weekly weekly newsletter devoted to SRE
- SRE at Google landing page for learning more about SRE in Google
- Komodor K8s Reliability learning center with resources for SREs working with Kubernetes
- SRE: What Do You Need To Know To Master This Role? resource list
