Proposal: DevOps

Unfortunately there is currently too much (negative) hype attached to the word devops. But there is also increasing interest into this field, as you can see new proposal are being opened on topics that would be a subset of this one (namely, continuous integration, chef, software quality assurance...). So I am trying to reboot this proposal, which already had many good questions here: http://area51.stackexchange.com/proposals/66151/devops

Trying to stay away from that (the hype), and simultaneously trying not to start the customary religion war between SOers doing sysadmin and SFers writing code, I can tell you what are the responsibilities of my work position, and why my questions would most possibly be off-topic on stackoverflow and serverfault (and several questions that are already in the proposed examples were real questions which had been closed). Sometimes because they are just off-topic, sometimes because opinion-based, sometimes just nobody will answer them. Note please that all of these examples do not define a different scope to the questions, but do address and require a different scope for the answers: so that a "give me options for task" question that would rejected on SO and SF would be well accepted here (if it is not blatantly lame, obvious).

development support

This spans from configuration of a code-review system, like gerrit, or static code analysis, and how to let these components interact. In part it's about procedures and guidelines, in other parts it is scripting, it often requires programming, and always basic sys-admin.

  • how to require at least three +1 CR votes in gerrit for approving a changeset?
  • how to let sonar cast votes against CR instead of V when integration test coverage is below a threshold?
  • what is a git repository setup that allows snapshots releases of components in the dev environment, and two differently versioned release branches for staging and production?

continuous integration

This boils down to integration testing configuration and reporting, and homogeneity of components build processes (well; a lot more actually, but I don't want to start my daily rant against agile here); would mostly be a sysadmin role against a build server, but requires very deep knowledge of build tools, the language, how to write good tests, and testing code in general.

  • how to classify maven-generated rpm packages of spring java components in nexus, for the different deployment environments?
  • functional/integration test failing with unit tests successful? how to report/what to do?
  • how to enforce semantic versioning in commits, accordingly to integration tests results?

continuous deployment

This is power-scripting helped/depending on very different, but very specific, tools (chef/puppet/rundeck/...). While it is true that the provisioning of standard services to service nodes is pure sys-admin (i mean tomcat/postgres/postfix/....); the same for in-house components composing a platform requires very strong understanding of a software architecture.

  • what is a good way to mark a nightly compose as a "good nightly", so that the most recent is used as a staging release-candidate when the sprint ends?
  • should deployment of development environment be triggered by the code review code repository, and staging/production from the canonical one, or every envronment from different branches in the canonical one? (note: this is primarily opinion-based on purpose)
  • how to setup the chef environment so that the httpd proxy load-balancer is updated when a new node of an horizontally scaling component is deployed?

monitoring

This is not monitoring of liveliness of a smtp server on a random linux distribution; it is the glue between monitoring facilities exposed by the deliverables and external monitoring tools. This need strong understanding of the software architecture, development skills for this specific duty, domain knowledge. Counter-intuitively, this would be most often closer to being on topic on programmers, then server fault.

  • how to write the xml configuration of an hyperic agent plugin for the mbean exposed by spring in this way?
  • what mechanism is available for dynamically changing the threading of an activemq queue consumer accordingly to the fact one broker of a two nodes failover setup failed?
  • how to write a EL query that returns this, given that setup of logstash tags?

In doing my daily job of build engineer, I ask myself hundreds of these questions. Sometimes stackoverflow fits, sometime server fault does (less often); anyway, it would be awesome to have a tool of the same utility and helpfulness of stackoverflow, to answer all the questions that would be marked off-topic, getting help and letting share my knowledge with others.

share
    
downvoter care to explain downvote? – guido Apr 2 '15 at 8:17
    
Seems repetitive of ServerFault – Mike Mackintosh Sep 4 '15 at 17:56
    
I agree with @MikeMackintosh. ServerFault seems like the appropriate space already. – J.Zimmerman Sep 8 '15 at 17:42
2  
Not in scope. While a sonar question about its ldap configuration may well be on topic in SF, I do not see it fit for how to stick sonar in a build pipeline, cause SF had nothing to do with software builds (SO does partially, but again with a different scope). – guido Sep 8 '15 at 17:49
3  
(CS Student perspective)... I feel like I would only go to Server Fault for things like asking how to build a physical server, how ECC Ram works, or questions about Data Center maintenance and the like. I want DevOps to succeed because I can ask more on the process flow of each stage of the development cycle for a piece of software and the tools used along the way. I think DevOps would refine the Server Fault site this way too (ex: questions on continuous integration should not go in Server Fault, Web Applications, Programmers, Stack Overflow or any other SE site). – willko747 Dec 27 '15 at 18:12
    
see also: stackoverflow.com/questions/41424058/… – J. Doe Jan 3 at 10:29
    
Funny, every time I see the term, "SLDC," I think of IBM's SDLC (Synchronous Data Link Control), 3174 controllers, and bus and tag cables. – Ron Maupin Feb 17 at 2:11
up vote 4 down vote accepted

The emergence of cloud or cloud-alike technologies modified the landscape of software development and deployment and brought new perspectives and challenges. Because we have the possibility to work with ephemeral servers, we want to take advantages of it. Instead of upgrading third-party software, we test and deploy on brand new, up-to-date, servers and ditch the old ones. Instead of carefully removing rootkits from compromised servers, we replace these servers, taking the compromised ones out of production for careful analysis, before ditching them. Instead of working with huge powerful do everything servers, we rely on dozens or myriads of smaller servers.

We need technical solutions

  • that work well with ephemeral resources.
  • that scale properly to handle an arbitrary amount of resources, we need to continuously adapt resources capacity to resources usage.
  • that scale properly to be applied at high frequencies, deployments can occur several times a week or a day.
  • that efficiently abstract resources to let us focus on services. (Efficiently does not mean totally!)

This perspective shift justifies the emergence of devops as a new discipline and e need to build a pertinent and easily accessible technical literature corpus considering problems from this very perspective. The Q & A format on SE seems appropriate, but this perpective shift also implies a shift in how we evaluate the relevance of answers to technical questions, and it does not seem that a SE-site where all devops themes fit already exists: these are two excellent reasons to found a dedicated community for us to share our knowledge and experience.

share

Given questions like this -> https://meta.stackoverflow.com/questions/294923/are-ansible-puppet-chef-salt-questions-on-topic on meta stackoverflow, I think it's pretty apparent that this is needed.

Switching from being a front-end Delphi programmer to a total back-end Linux guy, I'm seeing diminishing returns on crafting good questions on stackoverflow and even less on programmer stackexchange, where everything is off topic.

Unix and Linux is probably currently the best place to ask questions about most devops stuff, but I don't see why this site wouldn't include some of the fancier powershell/azure stuff that is permeating the ecosystem (at least the corporate ecosystem).

share
2  
Why would those questions not be on scope for Serverfault? – Mike Mackintosh Sep 4 '15 at 17:55
1  
Maybe it is, but I think ansible playbooks can be programming as well. We've got complete non-programmers who use it to stand up customer devices and complete non-network guys who use it to roll out changes to development environments. – Peter Turner Sep 4 '15 at 18:21
    
I'm of the opinion that "DevOps" related topics should be in ServerFault. Good systems guys are already programming even if it's not part of a particular product or project. The systems landscape is just shifting more heavily to rely on programming skills and automation. – J.Zimmerman Sep 8 '15 at 17:46
2  
@J.Zimmerman you could definitely ask there, but you'd probably be waiting for an answer. devops is on the bad tag list so you can't even tag it as such. So if devops can't even technically exist as a topic unto itself, it probably shouldn't be on serverfault. Unless it doesn't actually exist as a subject. – Peter Turner Sep 9 '15 at 3:36
1  
@PeterTurner - I guess that may be part of the problem. But in general "DevOps" is a little like "Cloud" in that it doesn't necessarily meaningfully classify something. Looking over some of the commentary about things on the bad tag list the argument to exclude "devops" is generally that there is probably a more specific tag or set of tags you can use (e.g. automation, continuous-integration, httpd, nginx, git, jenkins, powershell, puppet, etc...). – J.Zimmerman Sep 9 '15 at 20:53
1  
@J.Zimmerman, re: "Good systems guys are already programming" That may be true in your experience, but for novices like me with a Desktop IT background and whom are moving into Junior DevOps roles managing legacy software & servers it's not even remotely true. Having a forum dedicated to learning development for guys with an operations background it still very relevant. – user30031 Oct 12 '16 at 19:57

Here is my rationale for saying "yes".

  1. Solutions architecture is what I do. Devops is how I do it. I apply Devops methodologies in at least three areas: 1. Software engineering; 2. Operations e.g. systems engineering; 3, Security.

Currently, something like 70% of Devops individuals define Devops in terms of their work in software engineering, 30 % define it exclusively in systems engineering terms as Systems Reliability Engineers, and 0% or close to 0% don't define it at all in terms of their security work. In other words, the various definitions of Devops that have been bandied about are incomplete. And those practitioners of Devops are guilty of more than a little tunnel vision when they have been peddling their definition of Devops exclusively in terms of the limited scope of the work e.g. exclusively software engineering, etc. - that they do. And when they do that, they create no small amount of confusion among everybody else. This reminds me of a bunch of blind men touching the various parts of an elephant, and describing the elephant as anything but an elephant. I am very frustrated with these people and their act: It's time to describe the elephant as an elephant.

One more time: Devops is a methodology, and the methodology makes sense and is applied in specific contexts such as software engineering, systems engineering and security. Absent these contexts, Devops doesn't mean a damn thing.

We need a Devops site that defines Devops and the contexts of Devops accurately, and that has its act together.

  1. Currently, I can't think of a single site on SO where:

I can ask about how to build continuous integration pipelines.

Integrating Docker into continuous integration pipelines

Interfacing the components of a continuous integration pipeline

Discuss the pros and cons of various orchestration tools

Cloud operations

The different types of automated testing a Devops should know.

etc.

I am looking to build Big Data pipelines - that's probably part of my Devops obligation, too. As in, whatever it takes for a more efficient, more reliable workflow.

If you are a Devops and you never ran into workflow issues that didn't drive you crazy, then you aren't a Devops.

share
    
+1 for workflow issues that drive you crazy... – Michael B Jan 11 '16 at 22:28

You may be surprised but I don't really see a need for DevOps, I got very good help on SO for all my DevOps related questions, and at the same time very bad feedback from SF site, so you can imagine what I am currently using.

I don't know if my good experience is because of my high SO score, but the feedback was great, even if in many cases it was borderline.

Use you imagination to make the questions not sound subjective, and SO will give you the answers.

share
2  
I appreciate your answer points, but I do not think that having an answer or not in SO, is a sufficient justification for what is ontopic for the rules of it (or not). I upvoted you anyway, but, I think the "bad attitude" comment on SF should go. It is only primarily your opinion and does not count as facts. – guido Jan 20 '16 at 18:31
    
I had the exact opposite experience with being unable to use any exchange site to help me with my devop needs. I've had to use forums, irc, and blogs to get my questions answered. – GMNoob Dec 20 '16 at 12:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .