self-hosted web platform with centralized management that can scale?

[alyarb]

For the longest time, our web hosting business has been LAMP-based and run on standalone servers running ISPConfig3 for site management.

Most of our sites run WordPress, but we have recently gotten into more complex projects that run Django or Oscar. Some WP-based customers are putting more functionality on their roadmap and will be transitioning to Django in the future as well.

ISPConfig3 oddly doesn't have any support for mod_wsgi, and you especially have to hamstring/break it to get SSL working on a WSGI-based site running in ISPConfig. Creating/managing PHP sites are stupid simple on ISPConfig by comparison, and ISPConfig can be used to manage multiple sites running on multiple web servers.

We like many of these aspects of ISPConfig, but we need to move to something with better support for Python/WSGI, and putting the PHP and WSGI sites into containers should simplify development and deployment too. Ajenti has a nice admin panel, and greatly simplifies the creation of WSGI sites, but Ajenti cannot manage a cluster of web servers or containers.

We can't be the only ones facing these challenges. So how do you guys have your scaled-out web stacks setup, can you describe what you are running on and how the management works? Right now ours looks like this:

PHP or python site
ISPConfig3 or Ajenti
Apache or nginx
Debian 9
ESXi


We have 4 data centers and tons of under-utilized hardware, and web farming is not ---in any way--- the bread and butter of our business, so we want to make the most of our resources rather than paying for hosting on Digital Ocean or other providers. I'd love to hear what other admins are using to scale and manage their web servers and web sites. Can you have PHP and WSGI sites running side-by-side under a centralized, unified management umbrella?

For some applications, we are also looking at glusterfs and xtreemfs for file system clustering, and Percona for db clustering, but may be wishful thinking to incorporate that into this question.

Are there any admin tools that can consolidate the management of all these different layers onto a single pane, or am I way off-base? What do you guys use to meet these needs?

We can work in the shell, write and call scripts, and tha'ts all fine, but we also have some executives that like to get hands-on (maybe once a year!) and the warm/fuzzy feeling that comes from a little polish here and there goes a long way for us too.

Thanks for the thoughts.

 

[Red Squirrel]

I'm not aware of any such solutions but I had started coding one, then got side tracked and have not touched it in years. I'd probably end up rewriting it if I was to start it up again. But basically it was a web interface where you can easily setup all the normal stuff like virtual domains etc and the idea was that you can load the config up on another server and it would all be setup in one go. My future goal was to add clustering, so it's decentralized management and you can quickly move sites/configs between servers. It was geared as a replacement for Cpanel as that is super expensive. My program would have automated a lot of stuff too like security stuff. Perhaps it could even use SElinux. Half the battle with SElinux is learning to configure it so everyone just turns it off but my solution would just automate it all.

These are all things you'd think would already exist but anything that is close to a solution still requires a lot of setup and config before you even get it going. The goal with my setup would be fast deployment. If you have 100 servers to setup, you just install the base software, point it to an IP of an existing managed server and it would do the rest. Heck could even make it so you can do it from an existing server and point to the IP of a new server, and it would push the agent and software via SSH.

That said, for a commercial setup if they're willing to spend money, I'd probably look at Cpanel or WFM (I think that's what it's called?).

 

[alyarb]

Yeah, the lack of next-gen (or even modern) open source cluster orchestration really staggers me. Particularly an admin panel that can transcend all of your layers....really does not exist.

Again, ISPConfig does all that you describe, and is surprisingly great at what it does, babysitting all your apache configs and multiple servers and all that, but it only excels in PHP-based hosting.

I can't take Cpanel and WHM seriously because it's mediocre commercial software that doesn't take Debian or python seriously. This is such a bummer that we might be coming up with our own clunky interface in-house.