Administering NEOSYS Server: Difference between revisions

From NEOSYS Technical Support Wiki
Jump to navigationJump to search
 
(9 intermediate revisions by 3 users not shown)
Line 49: Line 49:


==Clearing File Locks==
==Clearing File Locks==
Due to many reasons, while a user is working on a document in NEOSYS, the window/system may hang and he might close the window. However NEOSYS doesn't know as to when the user has closed the window and locks the file and will release it after 5 minutes of consecutively not getting a response. The user meanwhile would get a message "Document is being updated by XXXX. Open for viewing only?".
Due to many reasons, while a user is working on a document in NEOSYS, the window/system may hang and he might close the window. However, NEOSYS doesn't know when the user has closed the window and locks the file and will automatically release it after 5 minutes of not getting a response. The user meanwhile would get a message "Document is being updated by XXXX. Open for viewing only?".


Sometimes, a user might keep a document open in a window and might lock & leave his workstation, and as such the file is locked by NEOSYS, till he releases the document. In this case also the user gets the above mentioned email.
If the document was being created and the user closed the window without saving the document, then the message would be "<document name> is being created by XXXX. You cannot view or update it until they have finished or cancel."
 
Sometimes, a user might keep a document open in a window and might lock & leave his workstation, and as such the file is locked by NEOSYS, till he releases the document. Even in this case, the user gets the message mentioned above.


Instead of going ahead with directly clearing the file locks, Support should follow the below procedure:
Instead of going ahead with directly clearing the file locks, Support should follow the below procedure:


NEOSYS Support should avoid doing manual work checking and clearing locks until at least 30 mins after the original event. This is because locks have a 10 minute AUTOMATIC expiry time and therefore cannot remain locked unless some window (perhaps hidden or lost on some workstation) is extending the lock time or there is an error with the server clock.
NEOSYS Support should avoid doing manual work of checking and clearing locks until at least 30 mins after the original event. This is because locks have a 5 minute AUTOMATIC expiry time and therefore cannot remain locked unless some window (perhaps hidden or lost on some workstation) is extending the lock time or there is an error with the server clock.


# Do not address the issue until at least 30 minutes have passed from original issue
# Do not address the issue until at least 30 minutes have passed since the original issue
# Get a second screenshot from the user if the problem continues
# Get a second screenshot from the user if the problem continues
# IF all workstations have been REBOOTED to ensure no hidden or lost windows holding the record open.
# IF all workstations have been REBOOTED to ensure no hidden or lost windows holding the record open.
# Get the List of Documents in Use
# Get the List of Documents in Use
# Check the Date and Time of the report to see what the server clock thinks the time it and if wrong, ensure that the server clock is updated from the internet automatically.
# Check the Date and Time of the report to see what the server clock thinks the time is and if wrong, ensure that the server clock is updated from the internet automatically.
# Check the "Date Time Locked" and "Date TimeLock Expires" columns for the offending record. This will tell you when the lock will expire and also indicate if it is being relocked by some hidden or lost window.
# Check the "Date Time Locked" and "Date TimeLock Expires" columns for the offending record. This will tell you when the lock will expire and also indicate if it is being relocked by some hidden or lost window.
# Draw some conclusion about why the lock APPEARS to remain in place from the above.
# Draw some conclusion about why the lock APPEARS to remain in place from the above.
Line 79: Line 81:


  DELETE LOCKS filename*key
  DELETE LOCKS filename*key
===Sample email to send to users who face "Document is being updated" message===
<pre>
This message means that the user is already editing this document on another window/tab of their browser.
If this document is not open on any browser window/tab of the user, then this message will appear if the user had closed the browser window/tab without Saving and Releasing/Closing the document he was editing. This lock gets automatically removed after 5 minutes, and the user will be able to edit the document again.
To prevent documents from getting locked, we always recommend that users periodically Save the document they are editing and click the "Release" button, or the Close button (or simply press Esc button on the keyboard), before closing the browser tab/window.
</pre>


== Scheduling a reboot of the server ==
== Scheduling a reboot of the server ==
Line 142: Line 153:
Client servers might experience unexpected shutdowns due to power failure, faulty UPS or other reasons at client site.  
Client servers might experience unexpected shutdowns due to power failure, faulty UPS or other reasons at client site.  


Unexpected shutdowns are noticed in the server System Logs usually after a failed backup or frequent server down trends on Nagios. Filter the system logs by event ID "6008" to view all unexpected shutdowns in the past and look for any pattern.
Unexpected shutdowns are noticed in the server System Logs usually after a failed backup or frequent server down trends on Nagios.


Support staff MUST proactively escalate this issue to Client IT immediately so that they take action and fix the issue which caused the unexpected shutdown. Email them the sample mail below along with screenshots as proof. CC every one from nightly backup list, so that client IT feels the pressure to fix things. Ensure that client IT confirms that the issue is resolved.
To view all the times that the server has unexpectedly shutdown (as shown in the screenshot below), in System Logs click Filter Current Log and enter event ID "6008".
 
[[File:Unexpected shutdown.png]]
 
 
Support staff MUST proactively escalate this issue to Client IT immediately so that they take action and fix the issue that caused the unexpected shutdown. Email them the sample mail below along with screenshots as proof. CC every one from nightly backup list, so that client IT feels the pressure to fix things. Ensure that client IT confirms that the issue is resolved.


As soon as Support re gains access to the server, do not start any processes on the server and make the Client wait until a proper check is carried out for any damage done.  
As soon as Support re gains access to the server, do not start any processes on the server and make the Client wait until a proper check is carried out for any damage done.  
Line 158: Line 174:
Dear Team,
Dear Team,


There was an unexpected shutdown on the NEOSYS server. Unexpected shutdowns is a very serious issue and can cause IRREPARABLE DAMAGE to  
There was an unexpected shutdown on the NEOSYS server. Unexpected shutdown is a very serious issue and can cause IRREPARABLE DAMAGE to files in NEOSYS. Fixing damaged files can be tedious, result in USER DOWNTIME and in the worst case force us to restore from backup, which can result in LOSS OF UP TO A DAY'S WORK BY USERS.
files in NEOSYS. Fixing damaged files can be tedious, result in USER DOWNTIME and in the worst case force us to restore from backup, which  
can result in LOSS OF UP TO A DAY'S WORK BY USERS.


If power failure is the issue, then the only solution is to buy a UPS with a CABLE and software that shuts down the server AUTOMATICALLY  
If power failure is the issue, then the only solution is to buy a UPS with a CABLE and software that shuts down the server AUTOMATICALLY when the UPS runs out of battery. This is the only correct and proper solution which everybody with sophisticated computer systems MUST use since power cuts are unavoidable.
when the UPS runs out of battery. This is the only correct and proper solution which everybody with sophisticated computer systems MUST use  
since power cuts are unavoidable.


Is there a UPS in place? Is the software that schedules shutdowns working properly?
Is there a UPS in place? Is the software that schedules shutdowns working properly?
Line 174: Line 186:
</pre>
</pre>


<b>Sample Screenshot</b>
==[[Procedures#Handling_.22List_of_Open_Ports.22_reports| Handling list of open ports report]]==
 
[[File:Unexpected shutdown.png]]

Latest revision as of 13:12, 28 February 2018

Starting NEOSYS Services

Automatic Ways

NEOSYS processes are configured to start early in the morning when the server restarts after a windows upgrade. We also schedule the process to start at windows startup in case of any unforseeable server restarts. Depending on which datasets and how many processes of that dataset are to be started are configured in the System Configuration File of NEOSYS, the appropriate number of processes will be automatically started once the initial dataset is started.

Manual Ways

Run the Schedule task which is configured to start the dataset required. Directly double click on the STARTDATASET.cmd file to start the required dataset.

Closing NEOSYS Services

Two automatic ways

  • NEOSYS services will automatically initiate a database integrity check, “backup” and close down at 2am by default.

To change the time of the automatic process do

F5 ED DEFINITIONS 
  • line 3 is the minimum time to start the backup and shutdown (default 7500 ie 02:00)
  • line 4 is the maximum time to start the backup and shutdown (default 7800 ie 02:05)
  • both times must be in seconds past midnight

Press F5 and type

PRINT ICONV(’02:00’,’MTH’) to calculate this easily
  • NEOSYS will automatically close down during the upgrade procedure.

Three manual ways

Warning: NEOSYS services do not shutdown automatically on server shutdown so you need to use one of these methods before shutting down or rebooting a server.

  1. To close individual services, press Esc three times on a service and then ctrl+C once or twice. Repeat for each service that you want to close.
    This can only be done if you can actually see the services on the screen but the services will not be visible if they were started by scheduled tasks while administrator was not logged in to the server … unless you login as /console with remote desktop.
  2. To close all services for one site/neosys installation, use the NEOSYS Support Menu – Stop/Restart NEOSYS
    Currently this has no option to stop the services for any other neosys sites/installations on the same server
  3. To close all services for all sites and installations of NEOSYS under a particular folder, use NEOSYS maintenance (or any other program) to create and then delete a GLOBAL.END file in the parent folder of all the neosys installations. eg if the installations are located in x:\hosts\clientcode\neosys etc then create and delete a file called x:\hosts\GLOBAL.END
    Procedure:
    • In NEOSYS Maintenance press F5 to get a NEOSYS command prompt
    • Type ED DOS ..\..\GLOBAL.END and press Enter
      (use ED DOS GLOBAL.END instead if you only want to close the services of the current site)
    • Press F9 to create the GLOBAL.END file
    • Wait until the processes close. Should be almost immediate.
    • Press alt+D twice to delete the GLOBAL.END file otherwise Neosys services will refuse to start

Restarting NEOSYS Services

To Restart NEOSYS services you should:

  1. First STOP the NEOSYS services on the server Stopping NEOSYS Services
  2. Then START the services again using instructions at Starting NEOSYS Services

Note - Restarting of Processes can be done when the users are logged into NEOSYS as it is not noticed by the user.

Handling unauthorised 3rd party software

See Removal of unauthorised 3rd party software

Clearing File Locks

Due to many reasons, while a user is working on a document in NEOSYS, the window/system may hang and he might close the window. However, NEOSYS doesn't know when the user has closed the window and locks the file and will automatically release it after 5 minutes of not getting a response. The user meanwhile would get a message "Document is being updated by XXXX. Open for viewing only?".

If the document was being created and the user closed the window without saving the document, then the message would be "<document name> is being created by XXXX. You cannot view or update it until they have finished or cancel."

Sometimes, a user might keep a document open in a window and might lock & leave his workstation, and as such the file is locked by NEOSYS, till he releases the document. Even in this case, the user gets the message mentioned above.

Instead of going ahead with directly clearing the file locks, Support should follow the below procedure:

NEOSYS Support should avoid doing manual work of checking and clearing locks until at least 30 mins after the original event. This is because locks have a 5 minute AUTOMATIC expiry time and therefore cannot remain locked unless some window (perhaps hidden or lost on some workstation) is extending the lock time or there is an error with the server clock.

  1. Do not address the issue until at least 30 minutes have passed since the original issue
  2. Get a second screenshot from the user if the problem continues
  3. IF all workstations have been REBOOTED to ensure no hidden or lost windows holding the record open.
  4. Get the List of Documents in Use
  5. Check the Date and Time of the report to see what the server clock thinks the time is and if wrong, ensure that the server clock is updated from the internet automatically.
  6. Check the "Date Time Locked" and "Date TimeLock Expires" columns for the offending record. This will tell you when the lock will expire and also indicate if it is being relocked by some hidden or lost window.
  7. Draw some conclusion about why the lock APPEARS to remain in place from the above.
  8. Do NOT clear locks if you can find nothing wrong;send info to and let programmers look at the issue

Commands to release the filelock/s are stated below:

To clear ALL file locks

F5
CLEARFILE LOCKS

To clear selective file locks

  1. NEOSYS Menu, Support, List of Documents in Use
  2. Maintenance mode (F5)
  3. Type the following command but replace "filename" and "key" with corresponding values from the List of Documents in Use
DELETE LOCKS filename*key

Sample email to send to users who face "Document is being updated" message

This message means that the user is already editing this document on another window/tab of their browser.

If this document is not open on any browser window/tab of the user, then this message will appear if the user had closed the browser window/tab without Saving and Releasing/Closing the document he was editing. This lock gets automatically removed after 5 minutes, and the user will be able to edit the document again.

To prevent documents from getting locked, we always recommend that users periodically Save the document they are editing and click the "Release" button, or the Close button (or simply press Esc button on the keyboard), before closing the browser tab/window.

Scheduling a reboot of the server

Prior to scheduling reboot

Send an email to the Client IT as well as NEOSYS Support staff, informing when the server is scheduled to reboot and the reason for the reboot.

How to schedule reboot of the server

Add a new scheduled task called "shutdown and restart" to run "one time only" at 3:30am the next day.

The program is called c:\windows\system32\shutdown.exe but you have to add the -r option in advanced properties to make the system restart as well.

This is useful for rebooting when software such as AVG antivirus requires a reboot.

Reboot1.jpg


Reboot2.jpg


Reboot3.jpg


Reboot4.jpg

Renaming a database name or code

A request to rename a database name or code may come from a client when they change their company name or break off an existing merger/partnership and if the old name exists in the database name or code which needs to be changed due to legal issues.

A database name is a non-unique identifier in a NEOSYS installation and is an antonym for the database code and is usually the legal name of the client. Renaming a database name involves compulsorily renaming the database code too. There isn't an escape from renaming the database code, however you can rename the database code back to the original one after you have done the procedure below.

A database code is a unique identifier per installation of NEOSYS. A complete note about database codes is available at Assigning database codes

  1. Renaming of a database has to be done when users have logged out of NEOSYS
  2. Take a backup of the database prior to the below steps
  3. Close all processes using the global.end method and do not rename until all steps are completed
  4. Open up Maintenance mode and login to the database you want to rename
  5. Go to General > Backup & Data Management > Create a new data set - from the drop down select the database you want to rename i.e. original database
  6. On the disk space requirements message, select YES
  7. Enter the name of the new database incase you are renaming the database name or else just type the original name here and hit enter
  8. Enter the new database code on the next screen and hit enter
  9. Select YES to start - it will start copying the existing database files to the new one
  10. Once complete, it will prompt you to login/switch to the new database, select Yes and from the dropdown select the new database and hit enter and login
  11. Once you have successfully logged in into the new database, go to General > Backup & Data Management > Delete a dataset - and select the old one and follow the instructions to delete it
  12. If you want to restore the database code to the original one, then follow steps 5 to 10 to create a new database with the original code.
  13. You can follow the same instructions if you want to rename the test database code as well
  14. Rename the global.end file to ensure that the processes will start again

After you have done the above steps to rename the database, ensure the following are completed (if you have renamed the database code back to the original one then you do not have to do the below steps):

  1. Rename the "start maindatabasecode.cmd" and "copy maindatabasecode to testdatabasecode.cmd" files to reflect the new database codes and test that they open up
  2. Delete the existing desktop shortcuts and re-create new one and test that they open up
  3. Reconfigure the Scheduled Tasks to reflect the new processes and test that they open up


Handling unexpected server shutdowns

Client servers might experience unexpected shutdowns due to power failure, faulty UPS or other reasons at client site.

Unexpected shutdowns are noticed in the server System Logs usually after a failed backup or frequent server down trends on Nagios.

To view all the times that the server has unexpectedly shutdown (as shown in the screenshot below), in System Logs click Filter Current Log and enter event ID "6008".

Unexpected shutdown.png


Support staff MUST proactively escalate this issue to Client IT immediately so that they take action and fix the issue that caused the unexpected shutdown. Email them the sample mail below along with screenshots as proof. CC every one from nightly backup list, so that client IT feels the pressure to fix things. Ensure that client IT confirms that the issue is resolved.

As soon as Support re gains access to the server, do not start any processes on the server and make the Client wait until a proper check is carried out for any damage done.

Proper checking MUST be done to look for any damaged files in the software by doing a chk.files and fix it in case of any damaged files.

Email the users about the downtime due to unexpected shutdown on the server and mention that Support needs to do maintenance on it to fix issues. This will force the users to get their IT fix this issue permanently so that next time their work does not get effected.

You need not mention the period of downtime so that Client also knows that the situation is serious and requires time to assess the damage. Start NEOSYS processes only after Support has tested for no damaged files in the software.

Sample Email to client IT

 
Dear Team,

There was an unexpected shutdown on the NEOSYS server. Unexpected shutdown is a very serious issue and can cause IRREPARABLE DAMAGE to files in NEOSYS. Fixing damaged files can be tedious, result in USER DOWNTIME and in the worst case force us to restore from backup, which can result in LOSS OF UP TO A DAY'S WORK BY USERS.

If power failure is the issue, then the only solution is to buy a UPS with a CABLE and software that shuts down the server AUTOMATICALLY when the UPS runs out of battery. This is the only correct and proper solution which everybody with sophisticated computer systems MUST use since power cuts are unavoidable.

Is there a UPS in place? Is the software that schedules shutdowns working properly?
Please update us on this issue.

Best Regards,
XXXX

Handling list of open ports report