Setting up and using remote support

Getting agreement of client IT staff to provide remote support

Letter to obtain agreement of client IT staff to provide remote support

Installing and configuring the server

Initial Connection to the server before setting up permanent remote connection

In case of a remote installation you need to get an initial connection to the server before you can setup Cygwin for a permanent remote connection. For this purpose you can either use your customised reverse connect UltraVNC SC file or the one-time run Teamviewer utility.

Do not use Microsoft Remote Desktop Client (RDP/RDC) on port 3389 at anytime to access the server from the internet since IT suppliers not aware of the situation often setup the initial administrator password to something obvious like "password" or even blank and in this case there is a good chance internet worms will discover the "open door" and install themselves before you get the chance to put a strong password.

Installing and configuring SSH

Installing Cygwin with OPENSSH

Watch out for non-intuitive steps like clicking "skip" to install something.

1. Read Avoiding Corrupt Cygwin Installations
2. ENSURE that you are logged in as the local (NOT DOMAIN) administrator
3. Run http://www.cygwin.com/setup.exe
4. Install from Internet
5. Root Directory: c:\cygwin
6. Local Package Directory: c:\cygwin.lib
7. Direct Connection
8. Download Site: http://mirrors.kernel.org (near the bottom) - If this does not show in the list, key in the URL in the field User URL and click on Add
9. You will now receive a Setup Alert. Click OK
10. Select Packages: Maximise window then click View once to get Full
11. Next to the package OPENSSH, click the word Skip (once!) to get version 4.4p1-1 or later
12. Next to the package NANO, click the word Skip (once!) to get the latest version available
13. Next to the package WGET, click the word Skip (once!) to get the latest version available
14. Next to the package RSYNC, click the word Skip (once!) to get the latest version available
15. Next to the package EMAIL, click the word Skip (once!) to get the latest version available
16. Next to the package WHOIS, click the word Skip (once!) to get the latest version available
17. Check the NEOSYS INSTALLATION CHECKLIST for any other packages to install like the above.
18. Click Next and complete the installation

Configuring and starting SSHD

Open the Cygwin icon to get a linux/bash command line and type:

Run the following commands:

chmod +r /etc/passwd
chmod +r /etc/group
chmod 777 /var

Thereafter start with the ssh configuration:

ssh-host-config

Then on the following options type:

Privilege                   –   YES
New local sshd account      -   YES
Install SSHD as a service   -   YES
Enter value of daemon       -   ntsec
Different name              -   NO
Create new privileged user  -   YES
Enter a password now        -   Set any random password and should not be the same as the neosys server (8 characters min)

At the command prompt type

net start sshd

Configuring SSHD to use a non-standard port number

This is necessary if the router cannot forward port 19580 --> 22 and we don’t want to open port 22 directly.

Capitalization is signification in cygwin/linux commands

open cygwin command prompt

cd /etc
chown administrator sshd_config
nano sshd_config (assuming that you have installed the NANO editor)
notepad sshd_config (incase you havent installed the NANO editor)

Move your cursor to Port 22 and change 22 to 19580. ctrl+x to save. On the confirmation type Y and on the next prompt hit enter.

chown system sshd_config
net stop sshd
net start sshd

To check that the connection to port 19580 is successful you can run the following test:

ssh -p 19580 administrator@localhost

You will be prompted to confirm the connection (say yes)

Now enter the system password to complete the procedure.

Changing ssh login from “Administrator” to “administrator”

Current NEOSYS policy to cater for recent versions of Cygwin is to rename the windows Administrator user to administrator to keep a consistent ssh login across all installations.

If you forget to do this before installing or upgrading Cygwin then you must to the following:

1. Rename “Administrator” to “administrator” in Windows
2. In a Cygwin console do:

mkpasswd > /etc/passwd

It should come back with nothing

Error while changing Cygwin port 22 to 19580

Error Message: Could not open file for writring: permission denied

Occurence: Sometimes When you edit the sshd_config file through NANO.

Solution: In SSH shell, follow these commands:

cp sshd_config ashwin_temp (copies sshd_config to a new file ashwin_temp)
rm sshd_config (deletes sshd_config)
cp ashwin_temp sshd_config  (copies ashwin_temp to sshd_config)

Incase it does not copy sshd_config to ashwin_temp, than check whether an ashwin_temp filename exists and delete it using the rm command.

Opening up ssh connections to additional source ip nos

Starting a NEOSYS process will automatically restrict cygwin ssh to accept connections from known NEOSYS company static ip numbers.

In the cygwin command line, insert a line in the list of allowable hosts

DO NOT ALLOW ALL OR GENERAL SSH ACCESS TO NEOSYS CLIENTS SERVERS WITHOUT GETTING PERMISSION *AND* INSTALLING EMAIL ALERTS FOR LOGINS AS DESCRIBED BELOW

nano /etc/hosts.allow

sshd: ALL

or a ip numbers or CIDR format

sshd 12.34.56.78
sshd 12.34.0.0/16

Setting up email alerts for cygwin ssh logins

Use http://www.cygwin.com/setup.exe to install "email" and "whois" packages

Insert the following script using cygwin command prompt.

NOTE! it@neosys.com to whatever you want.

cd /etc
nano sshrc

#!/bin/bash
#
#you configure this

ALERTEMAILADDRESS=it@neosys.com

#
#get the ip number without the ipv6 prefix
FROMIPNO=`echo $SSH_CLIENT|cut -f 1 -d " "|sed 's/::ffff://'`
#
#quit with no message if from a known host

if grep -x $FROMIPNO /etc/trustedipnos
then exit
fi

#
#get the host name by reverse lookup

FROMHOST=`nslookup $FROMIPNO|grep "name ="`

#
#get whois info about the login ip number

#and pipe it into the mail program
#"&" on the end creates a new process in order not to delay login

whois $FROMIPNO|\
email -q -f nl1@neosys.com -s "login $USER $FROMIPNO $FROMHOST" -r \
mailout.neosys.com -p 2500 $ALERTEMAILADDRESS&

Make sure that you configure the file permissions

chmod a+x sshrc

Inserted trusted ip nos.

cd /etc
nano trustedipnos

#sorry, ip ranges and cidr etc not accepted yet

#vm1.neosys.com for remote checking
85.17.154.105

#nl1.neosys.com
83.149.104.167

#nl2.neosys.com
85.17.154.66

#uk.neosys.com
78.143.212.191

#nl3.neosys.com
94.75.233.2

Make sure that you configure the file permissions

chmod a+x sshrc

Troubleshooting SSH: If SSH connects and then disconnects immediately without exchanging keys

The first time that NEOSYS runs, it automatically adds source ip number restrictions to the sshd remote support configuration in /etc/hosts.allow and /etc/hosts.deny. This is an important security procedure to allow connection to clients systems from NEOSYS ip numbers only. This process allows only local and known NEOSYS ip numbers to connect using SSH. Upgrading NEOSYS will add and/or remove allowable ip numbers as NEOSYS configuration changes.

It is possible that in some client network configurations incoming ssh connections will appear to be from the clients internal routers with an ip unknown to NEOSYS due to NAT configurations. Therefore ssh connections will be blocked unless specifically allow the local ip number or it is added into an upgraded version of NEOSYS.

NOTE: Therefore you must check that remote support via ssh works AFTER you have run NEOSYS once (maintenance mode).

1. Look in the Windows, Computer Management, System Tools, Event Viewer, Application
2. Search for entries from source "sshd", double click and look in the Event Properties, Description for ip numbers
3. Information type sshd entries will give the ip number of successful sshd connections.
4. Warning type sshd entries will give the ip number of failed sshd connections.
5. Find the ip number of failed connections.

Possible Problem 1 - Port mapping in router is using NAT

If the ip number of failed connections is some local ip number (of the router for example) then possibly the inbound port forwarding has been done with NAT and the source ip number has been lost. Therefore the NEOSYS ip restrictions are blocking ssh connections because they appear to be coming from an unknown ip number (ie that of the router)

Solution 1A

Change the router configuration to not use NAT and leave the genuine original source IP number

Solution 1B

DO NOT USE THIS PROCEDURE TO BREAK NEOSYS SECURITY

DO NOT GRANT ACCESS TO ANY IP OTHER THAN CLIENTS ROUTER IPS

Add the routers IP number to the list of authorised ip numbers in the cygwin hosts.allow file as follows:

nano /etc/hosts.allow

and add the line as follows ... but put the ip number of your router

sshd: allow 192.168.0.99

1. router - sadly using NAT instead of plain old port forwarding
2. if the router ip changes then neosys remote support will fail until this line is changed
3. NB do not grant access to 192.168.* etc since this allows local LAN viruses to attack

Troubleshooting sshd

You can run the sshd service interactively to see all messages instead of having to search logs/events etc.

Unfortunately this will not work the same as the normal windows sshd service unless you assume the identity of the sshd_server user. To assume the identity of the sshd_server user you will have to reset its password to something new (since we dont take a record of it during sshd-host-setup) AND ALSO place the new password in the logon properties of the sshd windows service.

su sshd_server
/usr/sbin/sshd -D -p 19580

Reinstalling SSHD if service fails to startup

1. Look in /var/log/sshd.log for errors
2. Delete the following users: sshd and sshd_server
3. Remove the sshd service at the cygwin prompt type cygrunsrv –R sshd
4. Do the above Configuration and starting SSHD step again

Note that you don't have to reinstall cygwin entirely, just sshd with the above steps.

Upgrading SSHD / Cygwin

NEOSYS relies on cygwin to provide secure network access and support various linux/unix services under Windows, mainly rsync for interoffice consolidation.

Just like MS Windows update, cygwin should be updated at regular intervals to close security holes discovered in the software by its authors. This is particularly important for cygwin's remote access service sshd since it is exposed to the internet although on a non-standard port.

TO DO: Find a simple way to learn about when cygwin upgrades sshd and/or when there are issues generally with sshd. Join the cygwin and sshd security news email lists.

To find out what versions of cygwin/sshd are installed at NEOSYS clients, look at Nagios, last column "Status Information":

http://vm1.neosys.com/nagios3/cgi-bin/status.cgi?servicegroup=neosys-ssh&style=detail%22SSH%20OK%20-%20OpenSSH_5.9%20%28protocol%202.0%29%22

eg

SSH OK - OpenSSH_5.9 (protocol 2.0) 

Upgrading Cygwin Without requiring a server reboot

NEOSYS normal remote server support connection uses cygwin/ssh. cygwin can be upgraded while in use but only if a reboot is performed and perhaps some cygwin packages reinstalled.

You can use:

• vnc server
• direct rdp connection
• directly on the server
• teamviewer started manually on the server

You cannot use:

• standard NEOSYS remote support connection using rdp/cygwin/sshd
• teamviewer started using a standard NEOSYS remote support connection.

Procedure:

1. If not already done, rename Windows “Administrator” user to “administrator” before upgrading

2. Stop all programs and services that use cygwin. (Only sshd is installed as standard on all NEOSYS clients)

net stop sshd
net stop cygwinrsync
net stop exim

3. Kill any remaining cygwin processes in task viewer. This will terminate any standard NEOSYS remote connections using cygwin/sshd/tunnelier.

sshd
cygrunsvr

4. Follow the usual cygwin installation procedure and it will upgrade all installed packages automatically.

• If you get a message "files in use" then click "retry" AFTER killing any running processes that use the cygwin1.dll. Search Google if you dont know how to do this, or follow the instructions in the next section which requires a reboot.

5. Start the NEOSYS remote connection service - cygwin/sshd, and any cygwin services stopped in step 1.

 net start sshd
 net start cygwinrsync
 net start exim

Upgrading Cygwin with server reboot

If not already done, rename Windows “Administrator” user to “administrator” before upgrading

Connect using usual NEOSYS remote support.

Follow the usual cygwin installation procedure.

If and when cygwin "says files in use" then at console command prompt then click "continue". NB "retry" will not work because your NEOSYS remote support uses files like cygwin1.dll that are being updated by cygwin.

If you have used the "continue" option then, towards the end of the cygwin installation process, you may get error messages similar to the one below. You can ignore them.

"the procedure point __ctype_ptr__ could not be located in the dynamic link library cygwin1.dll"

Finally, you may get a message "postinstall script errors". Copy this message so you know what packages have to be reinstalled.

Your list may vary! The list of packages is longer if the cygwin1.dll file has to be upgraded as this is an essential library file for all cygwin programs.

 Package: base-cygwin
 Package: coreutils
 Package: bash
 Package: terminfo
 Package: _update-info-dir
 Package: base-files
 Package: colordiff
 Package: man
 Package: terminfo0
 Package: vim
 Package: wget

Reboot the server

Reinstall Bash and check that you can connect using usual NEOSYS remote support.

• The login user name might be changed to "Administrator" instead of "administrator".
• If you cannot reconnect after rebooting then the following steps (in particular the cygwin sshd package) may have to be performed directly on the server directly or using the usual initial NEOSYS remote installation procedures that do not rely on cygwin/sshd.

Reinstall any problematic Cygwin packages

1. Select View: "Up to date"
2. "Keep" to "Reinstall" for the packages listed in the previous section.

Finally, check that you can run the ls command in a cygwin command prompt window.

Note - If you dont reinstall bash after rebooting then the bash prompt will be abbreviated to something different and there will be no response to any command entered.

How to uninstall/reinstall cygwin

With setup.exe (the installer file of cygwin) you can uninstall individual packages but not Cygwin.

Before you do this, make sure you have stopped the cygwin service (NET STOP SSHD), removed the sshd server (cygrunsrv -R sshd), deleted the users (sshd & sshd_server).

To uninstall Cygwin you have to run the following in DOS prompt:

rmdir /s /q C:\cygwin

You cannot delete the cygwin folder from Windows explorer due to a Access Denied error and this is the best way to uninstall cygwin.

Getting Ownership and Permissions Correct

Installation of cygrin under domain administrator account needs to be fixed as follows:

1. c:\cygin Properties, Security, Advanced
2. Change owner to: Administrators
3. Tick: Replace owner on subcontainers

After changing ownership of all cygwin folders to Administrators all ssh login will be blocked and you will get a windows application event log message. "root" actually means sshd's user which is sshd_server by default or can be found in the cygwin ssh windows services properties under log on

fatal: /var/empty must be owned by root and not group or world-writable.

Fix this in cygwin console as follows:

chown sshd_server /var/empty

Configuring Firewall/Router

You will have to port forward 19580 on the router to port 19580 on the neosys server. Some routers call port forwarding “port mapping” or “virtual servers”

It is BAD idea to simply open port 22 since an open port 22 attracts scanners/hackers like flies.

Also configure port forwarding of port 4430 to port 4430. This is for remote access via https if desired at a later date.

Configuring Specific Client Routers

Adline Dubai - CISCO PIX Firewall

Sonicwall Firewall Configuration

How to install ssh on port 19580 over vnc on port 19580

Install vnc on port 19580

connect on vnc

setup cygwin sshd on port 22

test you can login on port 22

ssh neosys@127.0.0.1

change sshd port to 19580 (but it wont start)

schedule a windows system reboot in 10 mins at windows command prompt

shutdown -t 600

change vnc port to 5900 (if will disconnect you)

wait for 10 mins and try to ssh login on port 19580

Changing user on Cygwin

On SSH command line:

ssh neosys@127.0.0.1 (where 'neosys' is the username)

Installing and configuring UltraVNC

Download UltraVNC from http://www.ultravnc.com

Run installation – all default options EXCEPT choose INSTALL AS A SERVICE, and START SERVICE.

((screenshot required))

Run ‘Service Helper’ under UltraVNC Server in case you don’t see vnc tray icon on the notification area.

Admin options should make the following changes:

1. Deselect – Enable Java (to increase security)
2. Select – Allow loopback connection (essential for connection via ssh)
3. Select - Loopback only (to increase security)

Installing and configuring the client

It is advisabke to use http over ssh rather than remote access IE inside vnc because the user interface is much faster alhough it might not be so bad using vnc to hosts in the same country

Installing Putty

http://the.earth.li/~sgtatham/putty/latest/x86/putty.exe

Download putty.exe > run

Configuring Putty

ssh tunneling local:127.0.0.1:1235-->remote:127.0.0.1:80 so you can do http://127.0.0.1:1235/neosys on your own computer

there are a few computers where it doesnt seem to like the above usually where the target server is on a none-standard port or not listening on 127.0.0.1

so sometimes you have to map local 127.0.0.1:8080 to remote 10.0.0.255:8080 or whatever the listening ip and port is

Source: 1234 Destination: 127.0.0.1:5900 then click Add

Source: 1235 Destination: 127.0.0.1:80 then click Add

Once tunnels are setup then return to Session tab in putty and save the options then click open to connect to the server. Login as administrator with the servers password.

Ultra VNC Viewer:

As the port mapping is complete, enter 127.0.0.1:1234 to login into the computer screen remotely:

Configuring Ultra VNC Viewer Listen mode to use a non-standard port

Troubleshooting Client

Cygwin login on servers connected to any domain controller

If NEOSYS is loaded on a server which is connected to any domain controller, it may require the local administrator password reset.

Resolving VNC Password lockout

If you connect to a client server through CYGWIN and enter the VNC password 3 times wrong in a row, VNC will block you.

As you have already entered the server through CYGWIN, you can reactivate the blocked VNC by using the following commands on the prompt:

net stop winvnc

net start winvnc

Remote Desktop Connection

Servers are normally not exposed to the internet so IT staff and suppliers are often not careful to use strong passwords and use things like "password" or blank.

Given the above, it is NEOSYS policy NOT to use remote desktop via direct access from the internet at all and especially not long term. This is to prevent worms from instantly discovering possible entry points - typically before NEOSYS can even begin to enforce strong administrator password.

If it is otherwise IMPOSSIBLE (difficult or inconvenient does NOT count as impossible!) to avoid using remote desktop protocol to the public internet then a simple and effective way of significantly increasing security is to change the remote desktop port from 3389 to something else e.g. 33890 as per NEOSYS convention.

Changing RDC port from standard to nonstandard

1. Start Registry Editor.
2. Locate and then click the following registry subkey:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\TerminalServer\WinStations\RDP-Tcp\PortNumber

1. On the Edit menu, click Modify, and then click Decimal.
2. Type the new port number, and then click OK.
3. Quit Registry Editor.