Remotely Accessing Offline Servers
This was a project given to me within the first three months of working as a Senior Developer for a pair of Data Centres.
Remote access to virtual servers is common nowadays. What isn’t common is doing this access though a hypervisor or remote access card, and with only a web browser to control it.
Windows Server 2016 in an Android browser
If you’re a business with a server in a Data Centre, what happens if your server goes down? Do you…
- Go and attend to it in person?
- Pay someone else to look at it?
- Wait for an Operative to be free?
- Get someone to reimage/reinstall it for you?
As you probably know, in an IT emergency, you can lose money first. None of the above options get you back to work quickly. You need a quicker and more effective solution.
You’re probably used to using software such as ‘Remote Desktop’ and controlling guest systems that way. You might have used other programs on Mac OS or Windows, or third-party tools. Many of the others are simply are based on the VNC protocol behind the scenes. Regardless, they all rely on both of these:
- Running a background service/daemon of some kind on the server.
- Installing guest software or using software that came packaged with your OS.
Neither of these is required for this emergency access.
As mentioned, you normally remotely access a virtual or physical server by running a service (a background program) on it. That means you can only operate the machine once it is booted — if it doesn’t boot, or doesn’t respond, then you might have to attend the machine in person. This is a tool designed to overcome those limits — invaluable in Data Centres with thousands of servers.
It uses a browser front-end, but doesn’t require you to install any plug-ins, or download any executables, as with many popular tools and services. It’ll happily run within your Sandboxed tab in Firefox or Chrome, and never needs access to your real computer.
With modern hypervisors such as Xen, or physical servers with add-on cards, it is possible to offer complete control to a system without any Ethernet connections, restore access to an unbootable system, or even live reinstall the operating system!
Linux + XFCE in a Windows browser
Unlike almost all my other projects, which I write from scratch, this one had already been started, and did have some basic functionality in place already. However, the work done was incompatible with the newer versions of the backend toolsets.
On a regular PC, if you have power, can see the screen, and have HIDs (Human Interface Devices), such as a keyboard/mouse connected via USB or PS/2, you can use the PC. The emergency remote access system works exactly on this principle. That means you never connect to the server itself.
It’s a bit like using a copy of VirtualBox, but with controls to a specific customer’s node in a Data Centre. You have access to:
- The power switch
- A virtual USB or PS/2 keyboard and mouse
- A streamed copy of the primary display
My changes were as follows:
- Rewrote the authentication plug-in to be more secure, which was the primary motivation for all the changes
- Fixed keyboard emulation sequences, like CTRL+ALT+DEL, so Windows Servers 2012/2016/2019 can log in
- Got newer versions of the client/server software working, and added faster WebP-tile support for supported browsers
- Added proper support for mobile devices
- Improved mouse pointer synchronisation when relative-pointing device drivers are used
- Changed the new interface to fit in with the company’s brand guidelines and themes
Linux + XFCE in an Android browser
The basic VNC protocol limits passwords to 8 characters, does not support usernames, and treats different screens/desktops to be on different TCP port numbers. This tool is more secure than VNC alone, because:
- All data is encrypted in an HTTPS session.
- The VNC protocol is not used externally.
- It has a username and password system rather than just a password and port number.
- You can set passwords of almost any length.
- Passwords are, usually, pseudo-randomly generated, and quite long.
- Decent logging, with ‘fail2ban’ anti-brute forcing compatibility.
The Usage Process
The tool works as follows:
- Invalidated tiles (rectangular changed areas) of the screen are then sent to the user’s browser, either as PNG, JPEG or, if the browser supports it, WebP images, along with the co-ordinates of the tile. This builds and updates the ‘screen’ within the user’s web browser.
- The user can tap on a text area to pop up their virtual keyboard, and then these key presses are sent to the target system’s keyboard buffer.
- Taps/clicks on the screen are sent as absolute co-ordinates. Relative co-ordinates are used instead if only PS/2 pointing devices are available.
- I also added ‘modifier’ shortcut keys, and other controls not present on mobile keyboards, which sit above the software keyboard.
Reimage and Reboot
In a separate interface, you can reimage and reboot servers on demand. This is handy if you shut down your server by accident or want to revert it back to an earlier version.
The SSH version of this tool allows you to connect to a physical or virtual server’s console with an SSH client. It only works with Linux.
I also was in charge of bringing this up to date, although the only changes that were needed were module upgrades, then the code was packaged and redeployed across the two Data Centres.
Windows Server 2016 in a Linux + KDE browser