Administering NEOSYS Server: Difference between revisions

From NEOSYS Technical Support Wiki
Jump to navigationJump to search
Line 163: Line 163:
Dear Team,
Dear Team,


There was an unexpected shutdown on the NEOSYS server. Unexpected shutdowns is a very serious issue and can cause IRREPARABLE DAMAGE to  
There was an unexpected shutdown on the NEOSYS server. Unexpected shutdown is a very serious issue and can cause IRREPARABLE DAMAGE to  
files in NEOSYS. Fixing damaged files can be tedious, result in USER DOWNTIME and in the worst case force us to restore from backup, which  
files in NEOSYS. Fixing damaged files can be tedious, result in USER DOWNTIME and in the worst case force us to restore from backup, which  
can result in LOSS OF UP TO A DAY'S WORK BY USERS.
can result in LOSS OF UP TO A DAY'S WORK BY USERS.

Revision as of 07:24, 30 October 2016

Starting NEOSYS Services

Automatic Ways

NEOSYS processes are configured to start early in the morning when the server restarts after a windows upgrade. We also schedule the process to start at windows startup in case of any unforseeable server restarts. Depending on which datasets and how many processes of that dataset are to be started are configured in the System Configuration File of NEOSYS, the appropriate number of processes will be automatically started once the initial dataset is started.

Manual Ways

Run the Schedule task which is configured to start the dataset required. Directly double click on the STARTDATASET.cmd file to start the required dataset.

Closing NEOSYS Services

Two automatic ways

  • NEOSYS services will automatically initiate a database integrity check, “backup” and close down at 2am by default.

To change the time of the automatic process do

F5 ED DEFINITIONS 
  • line 3 is the minimum time to start the backup and shutdown (default 7500 ie 02:00)
  • line 4 is the maximum time to start the backup and shutdown (default 7800 ie 02:05)
  • both times must be in seconds past midnight

Press F5 and type

PRINT ICONV(’02:00’,’MTH’) to calculate this easily
  • NEOSYS will automatically close down during the upgrade procedure.

Three manual ways

Warning: NEOSYS services do not shutdown automatically on server shutdown so you need to use one of these methods before shutting down or rebooting a server.

  1. To close individual services, press Esc three times on a service and then ctrl+C once or twice. Repeat for each service that you want to close.
    This can only be done if you can actually see the services on the screen but the services will not be visible if they were started by scheduled tasks while administrator was not logged in to the server … unless you login as /console with remote desktop.
  2. To close all services for one site/neosys installation, use the NEOSYS Support Menu – Stop/Restart NEOSYS
    Currently this has no option to stop the services for any other neosys sites/installations on the same server
  3. To close all services for all sites and installations of NEOSYS under a particular folder, use NEOSYS maintenance (or any other program) to create and then delete a GLOBAL.END file in the parent folder of all the neosys installations. eg if the installations are located in x:\hosts\clientcode\neosys etc then create and delete a file called x:\hosts\GLOBAL.END
    Procedure:
    • In NEOSYS Maintenance press F5 to get a NEOSYS command prompt
    • Type ED DOS ..\..\GLOBAL.END and press Enter
      (use ED DOS GLOBAL.END instead if you only want to close the services of the current site)
    • Press F9 to create the GLOBAL.END file
    • Wait until the processes close. Should be almost immediate.
    • Press alt+D twice to delete the GLOBAL.END file otherwise Neosys services will refuse to start

Restarting NEOSYS Services

To Restart NEOSYS services you should:

  1. First STOP the NEOSYS services on the server Stopping NEOSYS Services
  2. Then START the services again using instructions at Starting NEOSYS Services

Note - Restarting of Processes can be done when the users are logged into NEOSYS as it is not noticed by the user.

Handling unauthorised 3rd party software

See Removal of unauthorised 3rd party software

Clearing File Locks

Due to many reasons, while a user is working on a document in NEOSYS, the window/system may hang and he might close the window. However NEOSYS doesn't know as to when the user has closed the window and locks the file and will release it after 5 minutes of consecutively not getting a response. The user meanwhile would get a message "Document is being updated by XXXX. Open for viewing only?".

Sometimes, a user might keep a document open in a window and might lock & leave his workstation, and as such the file is locked by NEOSYS, till he releases the document. In this case also the user gets the above mentioned email.

Instead of going ahead with directly clearing the file locks, Support should follow the below procedure:

NEOSYS Support should avoid doing manual work checking and clearing locks until at least 30 mins after the original event. This is because locks have a 10 minute AUTOMATIC expiry time and therefore cannot remain locked unless some window (perhaps hidden or lost on some workstation) is extending the lock time or there is an error with the server clock.

  1. Do not address the issue until at least 30 minutes have passed from original issue
  2. Get a second screenshot from the user if the problem continues
  3. IF all workstations have been REBOOTED to ensure no hidden or lost windows holding the record open.
  4. Get the List of Documents in Use
  5. Check the Date and Time of the report to see what the server clock thinks the time it and if wrong, ensure that the server clock is updated from the internet automatically.
  6. Check the "Date Time Locked" and "Date TimeLock Expires" columns for the offending record. This will tell you when the lock will expire and also indicate if it is being relocked by some hidden or lost window.
  7. Draw some conclusion about why the lock APPEARS to remain in place from the above.
  8. Do NOT clear locks if you can find nothing wrong;send info to and let programmers look at the issue

Commands to release the filelock/s are stated below:

To clear ALL file locks

F5
CLEARFILE LOCKS

To clear selective file locks

  1. NEOSYS Menu, Support, List of Documents in Use
  2. Maintenance mode (F5)
  3. Type the following command but replace "filename" and "key" with corresponding values from the List of Documents in Use
DELETE LOCKS filename*key

Scheduling a reboot of the server

Prior to scheduling reboot

Send an email to the Client IT as well as NEOSYS Support staff, informing when the server is scheduled to reboot and the reason for the reboot.

How to schedule reboot of the server

Add a new scheduled task called "shutdown and restart" to run "one time only" at 3:30am the next day.

The program is called c:\windows\system32\shutdown.exe but you have to add the -r option in advanced properties to make the system restart as well.

This is useful for rebooting when software such as AVG antivirus requires a reboot.

Reboot1.jpg


Reboot2.jpg


Reboot3.jpg


Reboot4.jpg

Renaming a database name or code

A request to rename a database name or code may come from a client when they change their company name or break off an existing merger/partnership and if the old name exists in the database name or code which needs to be changed due to legal issues.

A database name is a non-unique identifier in a NEOSYS installation and is an antonym for the database code and is usually the legal name of the client. Renaming a database name involves compulsorily renaming the database code too. There isn't an escape from renaming the database code, however you can rename the database code back to the original one after you have done the procedure below.

A database code is a unique identifier per installation of NEOSYS. A complete note about database codes is available at Assigning database codes

  1. Renaming of a database has to be done when users have logged out of NEOSYS
  2. Take a backup of the database prior to the below steps
  3. Close all processes using the global.end method and do not rename until all steps are completed
  4. Open up Maintenance mode and login to the database you want to rename
  5. Go to General > Backup & Data Management > Create a new data set - from the drop down select the database you want to rename i.e. original database
  6. On the disk space requirements message, select YES
  7. Enter the name of the new database incase you are renaming the database name or else just type the original name here and hit enter
  8. Enter the new database code on the next screen and hit enter
  9. Select YES to start - it will start copying the existing database files to the new one
  10. Once complete, it will prompt you to login/switch to the new database, select Yes and from the dropdown select the new database and hit enter and login
  11. Once you have successfully logged in into the new database, go to General > Backup & Data Management > Delete a dataset - and select the old one and follow the instructions to delete it
  12. If you want to restore the database code to the original one, then follow steps 5 to 10 to create a new database with the original code.
  13. You can follow the same instructions if you want to rename the test database code as well
  14. Rename the global.end file to ensure that the processes will start again

After you have done the above steps to rename the database, ensure the following are completed (if you have renamed the database code back to the original one then you do not have to do the below steps):

  1. Rename the "start maindatabasecode.cmd" and "copy maindatabasecode to testdatabasecode.cmd" files to reflect the new database codes and test that they open up
  2. Delete the existing desktop shortcuts and re-create new one and test that they open up
  3. Reconfigure the Scheduled Tasks to reflect the new processes and test that they open up


Handling unexpected server shutdowns

Client servers might experience unexpected shutdowns due to power failure, faulty UPS or other reasons at client site.

Unexpected shutdowns are noticed in the server System Logs usually after a failed backup or frequent server down trends on Nagios.

To view all the times that the server has unexpectedly shutdown (as shown in the screenshot below), in System Logs click Filter Current Log and enter event ID "6008".

Unexpected shutdown.png


Support staff MUST proactively escalate this issue to Client IT immediately so that they take action and fix the issue that caused the unexpected shutdown. Email them the sample mail below along with screenshots as proof. CC every one from nightly backup list, so that client IT feels the pressure to fix things. Ensure that client IT confirms that the issue is resolved.

As soon as Support re gains access to the server, do not start any processes on the server and make the Client wait until a proper check is carried out for any damage done.

Proper checking MUST be done to look for any damaged files in the software by doing a chk.files and fix it in case of any damaged files.

Email the users about the downtime due to unexpected shutdown on the server and mention that Support needs to do maintenance on it to fix issues. This will force the users to get their IT fix this issue permanently so that next time their work does not get effected.

You need not mention the period of downtime so that Client also knows that the situation is serious and requires time to assess the damage. Start NEOSYS processes only after Support has tested for no damaged files in the software.

Sample Email to client IT

 
Dear Team,

There was an unexpected shutdown on the NEOSYS server. Unexpected shutdown is a very serious issue and can cause IRREPARABLE DAMAGE to 
files in NEOSYS. Fixing damaged files can be tedious, result in USER DOWNTIME and in the worst case force us to restore from backup, which 
can result in LOSS OF UP TO A DAY'S WORK BY USERS.

If power failure is the issue, then the only solution is to buy a UPS with a CABLE and software that shuts down the server AUTOMATICALLY 
when the UPS runs out of battery. This is the only correct and proper solution which everybody with sophisticated computer systems MUST use 
since power cuts are unavoidable.

Is there a UPS in place? Is the software that schedules shutdowns working properly?
Please update us on this issue.

Best Regards,
XXXX