What is required for developers to run their own VM server in an enterprise environment [closed]

Solution 1:

First of all, I do see a few reasons why your admins are right to push back:

• IT is also responsible for reporting on things like patch management, antivirus software, pci compliance, annual (or more frequent) security audits, etc. It's not just a matter of having your assurance that this is taken care of, but of also being able to prove it to outsiders.

  As an example, I run the network at a small college, and we have a physics lab with a few data collections machines for student experiments. The only thing they do is collect data from a scientific instrument and print out the results (to a directly attached printer) for the student to analyze and hand in to the instructor. They are never on the internet — even AV and Windows updates are applied via the local network. The only reason they are connected to the network and run AV software at all is the explicit purposes of reporting to my monitoring software that they still exist and are current. It's silly, as they would in reality be more secure to remove the network connection, but they were first paid for with an education grant and so those are my reporting requirements.

• Like it or not, your dev server is a production system from the standpoint of the developers. Give it maybe a month, and developers will have a hard time getting work done if it goes down because of processes you will set up that assume the server is available. Avoiding/limiting situations where workers are set idle because of technology failures is a big reason businesses still use centralized IT departments.
• If "ESX Server is the Enterprise Standard", you need to follow that. At this time, there are significant differences between how hyper-v, vmware, xen, and others do things, and a machine built for one can't just be assumed to run well on the other. If you are going to do this, IT is going to need to help manage it at some point, and they don't want to have to convert it over to vmware after there's a bunch of cruft on the machine that might cause a problem.
• Someday this machine will get old, and either need more regular maintenance or setting up a standard replacement cycle. Even new servers sometimes have bum parts. This situation almost always lands in IT's lap only after something has broken that prevents people from doing their jobs. By taking responsibility for the server early, IT can do a much better job making sure you avoid unplanned outages down the road.
• This one's personal, but I can tell you that the very last thing I want around my network is another desktop masquerading as a server. I've dealt with enough of those the last couple years to last me a lifetime.

That said, IT needs to be able to support this initiative. It's not enough for them to just say, "No". Challenge them to come up with an alternative that meets your (very real) needs. The only political situation here should be that their alternative is likely to have a bigger sticker price (because they're planning for costs that you can't see yet), and so the question will be who has to pay for it. IT won't want to because they didn't budget for it, but you'll balk because it's 6 times what you spent for a solution that you were happy with (for the moment).

Also, it sounds like you might be trying to run before you can walk. You want to revamp your development process. As a former developer, I think that's great. But don't just throw out a bunch of VMs and "environments" (ie: dev, stage, qa, etc). Plan out what the new processes will look like — how developers will get work done. Will you use continuous integration? Automated builds? Using what software to support them? Will developers be allowed to move code to production or staging, or will only QA have that ability? Do you need a separate staging? What about two dev branches (one for vNext, one for bugs with vCurrent)?

You might need a server just so the development lead or manager can work all that out, but if so then that needs to be the first step, and the setup and initial process design needs to be done before developers get their hands on it for actual use.

Solution 2:

1) What arguments have your developers made that won you over to allow these types of silos to exist within enterprises which have standard network and security policies in place that would generally (and understandably) preclude this type of non-(centrally)-managed infrastructure?

None - mostly because I don't play a management role in my organization so any of these "political things" don't really involve me. The only argument that would really persuade me is to allow something that is explicitly against the network policy and exempted from both the system operations team control and visibility is an air-gap and a CYA letter from my boss's boss.

It's not that I really want to be a the business of saying "No", it's just that there's no good way for this to end well from the operation team's standpoint.

1. We end up with servers managed by people who's primary skill set isn't managing servers on the network who don't have visibility of the entire network and its associated problem space. This is not just a political concern of "protecting" turf; as a trite example imagine what happens when a developer turns on DHCP for some reason.
2. We end up managing the development team's servers for them. This is messy for the opposite reason. The developers are constantly annoyed that we're patching this (breaking something that they know about but we don't), or their fighting for us to enable features that for a variety of good reasons we don't want enabled. This quickly turns into a deadlock where the operations team feels burdened and harassed and the development team feels frustrated and ignored.
3. There are political consequences - because then you have to explain to another department why the developers are "special" and why they get exempted from network policy.

2) Is this just a matter of the developers making a technical or business justification and ensuring that patch management and AV is going to happen - or more of a political struggle for control and ownership?

I don't think the developers need to make a business case - it's pretty clear that developers got to develop and in order to do that that they need a development environment of some kind. As for patch management and AV - that's the operations team job and we will ensure that it is done. It's not that we think developers can't do it. It's just we don't trust you do it right - System Administrators stay System Administrators because they don't trust anything to do anything right so it's nothing personal. Of course there is an obvious political issue of feeling like someone else is "doing your job" but that's not really a technical problem and hence is outside of SF's scope.

3) Given the choice, would you prefer to take ownership and support of the hardware/OS while giving devs local admin rights, or let them manage it entirely, while ensuring that they institute patch management/AV and charging them with responsibility should they cause problems?

Neither for the reasons outlined above.

4) If you successfully blocked developers from having local control of "rogue servers" on your infrastructure, did the developers just make due or did they (or you) move the development environment to a disconnected VLAN/entirely separate network?

Air-gap. The best way to handle this situation is to give the developers their environment (and control of it) and then air-gap or use some other robust method to separate it from the network. This is essentially how we handle public wifi. You want wifi services? Sure. You pay for the network connection, we'll manage the WAPs, but it'll never touch our network. Sorry. Your needs are but one of hundreds. There are other concerns we have to consider.

You don't want to be in the business of saying no, because developers (who are particularly technically clever) will find ways to get what they want anyway. So provide them an environment that suits their needs, make them happy and then find some way to prevent anything they do in development environment from touching the rest of your business network.

TL;DR: I would give you a server with whatever virtualization platform you wanted on either a separate physical network or VLAN. Access to your development environment would be through a single bastion host that the operations team would control and monitor. What you do with it your business - It won't be supported but we'll provide advice and help as time permits for the server administration side of things.

Solution 3:

If you came to me with a workstation class machine loaded to the hilt with consumer grade RAM, consumer-grade HDD's, consumer-grade PSU and consumer-grade RAID, I would refuse to put it on the server network too.

There's a lot you need to understand about putting something like that on server VLAN.

1. The server VLAN may very well be a DMZ. In a DMZ you do not put anything that is not hardened and secured. This is just a machine you handed them, they've no idea what you did with it. It also means regular patching and updates, which means being centrally managed. I'm sure as hell not going to log onto each un-managed server and apply patches by hand.

2. The components in that machine are going to fail. I promise. Within 6 or 12, 24 months, it's going to go belly up. Then, where are the backups? Oh, you didn't set them up? But, I thought it was a server? Oh, its a server that someone else provisioned?... and the blame game starts again

3. Who is going to take responsibility when it crashes and shit hits the fan? In most organisations "I gave it to the dev's to look after" is not going to fly.

4. Where are they going to put it? Servers are all rack mounted these days and putting a tower in a rack wastes space and their racks may not be designed for it.

So, the IT department are very justified in not putting this random computer on their server network.

However it is the IT departments job to ensure that YOU can do YOUR job properly. They need to make sure that you have what you need, when you need it. If you have a piece of software that the business needs to keep running, they have to provide a platform for it to run on. That's their job description. But you need to make sure that they have the information they need to do their jobs.

If you had come to me, in my organisation, told me you're starting a new project, I would have given you three VM's: Dev, Live and Staging. You would have full admin rights to Dev, and we would discuss what you needed to do your job for the other two. If you needed full admin rights to them, and could justify it, then you would get it. We have our VM deployment down-pat. VMWare makes this incredibly simple - it only takes about 5 minutes per VM to get it deployed.

It sounds like your IT department suffers from what pretty much every IT department in a large company suffers from. Building little castles and defending them with your life, not letting others in, being bossy, etc. As someone who deals with other people's IT departments every day, I see it all the time. And it's frustrating.

The basic fact is though that the change needs to happen from within the IT department, and it has to be initated from above them. And if you can get IT to realise that they are not a force unto themselves (as that most of them do not generate income for their businesses this can be quite a slap in the face), and that they are there to support the existing staff and enhance the business, then you'll find that your questions become irrelevant, because everyone will be playing happy families.

Solution 4:

Why do you want to get it added to the domain? To put it another way, which answers the question better : You can set up a lab to do whatever the hell you want, as long as it's not connected to the corporate LAN. (If you needed internet access, maybe you can get a DMZ-ed VLAN; that shouldn't be a problem, especially if you're only using it to go out, like for downloads.)

That is one, of many many many, different answers to the question.