Procedures

From NEOSYS Technical Support Wiki
Jump to navigationJump to search

Here are procedures to be followed by Support Staff in respect to various technical matters in day to day operations of client issues.

NEOSYS Support Policies

In order to maintain good payment speed by clients NEOSYS needs to restrict support to clients that dont pay their bills on time, however the degree of restriction needs to depend on an intimate knowledge of the client which cannot be expected from all NEOSYS support staff. Therefore we will use a simple escalation policy as follows:

Stop Support List

NEOSYS SUPPORT MANAGERS WILL maintain an overdue/stop list on a whiteboard visible to all support staff. Generally clients will go on the list immediately when their invoice is overdue and come off only after satisfactory commitment to pay have been obtained.

NEOSYS SUPPORT STAFF WILL discretely refer any calls for support from client on the overdue/stop list to support managers for handling.

Managers may well instruct support to provide support on a case by case basis even if clients are on the stop list. Being on the stop list does not necessarily indicate a major issue with accounts

In case clients pressurize neosys staff to provide support then the support staff must inform them that there is an issue with their account and that their request had been forwarded to the manager.

This policy excludes dealing with server failures, backup failures and measures required to be taken to fix these failures.

Client Password Policy

All client user passwords, including their initial one, are to be obtained via the user's email address using the password reminder/reset button on the login screen.

NEOSYS staff should never know users passwords therefore NEOSYS will not obtain and grant user passwords.

Support requests from ordinary client users

Any support requests concerning inability to obtain passwords will be forwarded to known skilled users on the client staff since this is the most efficient (not fastest) way to handle such issues.

Support requests from senior client management

Any support requests concerning inability to obtain passwords by senior client management users shall be handled directly by NEOSYS support staff in any way convenient to resolve the issue in the quickest possible time rather than the most efficient.

Bearing in mind that NEOSYS staff should never know user's passwords this will probably involve NEOSYS staff using the Password Reminder/Reset button to send a new password to the user.

User Defined Passwords

NEOSYS will provide user defined passwords in very special cases which must be pre-approved case by case by NEOSYS management. For many reasons, NEOSYS will not approve.

Currently this permission has only been granted to one NEOSYS client with several hundred of databases.

Handling client issues

Handling letterhead change requests

Support staff should reject any requests that requires the letterhead to be setup on the TESTING dataset before it is setup in the MAIN dataset.This is to reduce double work for support staff and to ensure that clients have a clear understanding of their requirements and also send the correct logo image. The MAIN dataset can be copied to the TEST dataset for any kind of testing.

Handling error messages

Important: Before Attempting to resolve client issues, please ensure that we have secure access to the NEOSYS server.

  1. The very first step is understanding client problem.
  2. Ask the client what error does he gets on the screen.
  3. If error seems to be familiar then resolve it over the phone.
  4. If error is unknown then ask user to send a screenshot of the error displayed along with the options used (basically you need to know HOW to replicate the error)
  5. Upon receipt of the error, check in all the wikis for a solution.
  6. If the issue is unknown or you don’t understand it clearly ask the user then use remote support to gain access to the users desktop to view how to replicate the error.

If it is new issue then report by escalation the same issue to your manager with a brief explanation.

NEOSYS support for resolving issues with totals on reports

If a client has a problem with any total output by NEOSYS software then NEOSYS support will advise them which other NEOSYS report or reports provide a complete breakdown of the total (if necessary, to individual transactions) and ask the client to locate any offending transactions themselves.

NEOSYS support staff will handle any issues where the total on the breakdown report does not add up to the total on the summary report.

Reconciling totals can be hard if there are many transactions involved. Regardless of how hard it may be, reconciliation is an operational task for users not for support staff since NEOSYS support staff will not get involved in understanding client transactions or data.

Using Support Tools

NEOSYS Software

The NEOSYS Software is currently supported on the following Operating Systems (OS) and browsers:

  • For Media & Finance modules, Internet Explorer 6+ on Windows
  • For Jobs & Timesheets modules, Internet Explorer 6+ on Windows or Safari 5.0 or Firefox on Mac

Users are to ensure that pop-up blockers and any 3rd party toolbars are deactivated/switched off or else certain pages and alert messages while using NEOSYS do not appear as a result of blocking from either the pop-up blocker or toolbars with built-in pop-up blockers.

NEOSYS Support personnel should additionally ensure that under Internet Explorer > Tools > Internet Options > Advanced > Browsing - the items Disable script debugging (Internet Explorer) and Disable script debugging (Other) are UNTICKED. This is because if NEOSYS generates any javascript error message, the same would disappear in the bottom left corner of a window, which in turn helps the programmer fix the error.

Note- The above mentioned process must be carried out after every Factory Reset.

Website Live Support

www.neosys.com is equipped with a Live Support software and clients can visit the website, click on this link and chat with any of our support staff, without the need for any installation. The client has to fill in their name and email address to connect to an available support personnel. During non-working hours, the Live Support icon on the website automatically displays "offline".

NEOSYS Support personnel who are authorised to provide such support, need to download a software called Kayako Live Support from http://hotfix.kayako.com/latest.php?product=lr&platform=win32&buildtype=stable

Once downloaded the account needs to be setup as follows:

  • Account Name: (as provided by NEOSYS IT)
  • SupportSuite URL: http://support.neosys.com/
  • User Name: (as provided by NEOSYS IT and usually same as Account Name)
  • Password: (as provided by NEOSYS IT)

The first time the account is setup, you need to close Kayako Live Support completely and restart for it to login and work properly.

Teamviewer

Since teamviewer allows no restriction on access once a fixed pass is installed, we cannot install fixed pass on teamviewer however convenient it might be.

RULE: NO FIXED PASS TO BE INSTALLED ON TEAMVIEWER IN ANY NEOSYS OR NEOSYS CLIENT COMPUTER

Running teamviewer live from a web link is fine because it does not allow installation of a permanent password

Documenting Processes in Wiki

How to create templates in wiki

Handling Nagios Client Monitoring system

Nagios is configured to display information pertaining to all NEOSYS client's server statuses which include multiple services such as:

  1. HTTPS: Most of NEOSYS clients are configured to have external web access via secure HTTP protocol (port 4430) from outside office. Nagios is configured to check port 4430 on a regular interval of 10 minutes and display any issues in accessing the same.
  2. SSH: As part of the support contract, NEOSYS should have external secure access to the client server usually over port 19580. Nagios is configured to check this port on a regular interval of 10 minutes and display any issues in accessing the same.
  3. Ping: Nagios is also configured to ping the client router as a measure to check if router responds incase the NEOSYS server is down.
  4. NEOSYS: This service works in a reverse direction, and the NEOSYS installation on the client server sends information such as databases running, current backup status, internal and internet IP addressess etc to Nagios on a regular interval of 10 minutes.

Some key information about Nagios is as follows:

  • Nagios is also configured to display information related to internal servers.
  • Clients hosted on a NEOSYS cloud server might not have services such as SSH or PING as this is monitored as part of the internal server service.
  • Nagios sends out email alerts to support2@neosys.com (which is forwarded to support@neosys.com) from 8 am to 12 midnight on all Dubai working days (Sun-Thu). No alerts are sent out on Fri and Sat, unless they are for NEOSYS internal servers.

NEOSYS support staff on duty has to follow the below outlined procedures in case of any Nagios items showing a critical or warning message for any service:

  1. Nagios is required to be checked first thing in the morning and any critical or warning messages need to be dealth with to resolve the same at the earliest.
  2. Some of the messages could be related to backup failures and the usual procedure as stated in #Backup Procedures needs to be followed. In case the backup issue isn't resolved by 9:30 am, the Nagios service needs to be scheduled with downtime for a minimum of 2 hours and maximum until 1 am next day if the issue cannot be solved.
  3. In case any HTTPS, SSH or PING service is down, immediate action is required and the relevant IT people at the client side needs to be contacted to get this resolved. A downtime of 2 hours is required to be scheduled with further intervals of 2 hours incase this is not resolved. Support staff shouldn't schedule downtime till 1 am next day, just to get rid of the alerts for the day. Proactive follow up with the client is required to get this resolved before the business day - more so, if there is a weekend ahead.
  4. In case the HTTPS, SSH or PING service goes down during the day, a grace period of 20 minutes is given before the issue is reported to the client IT. This helps incase there is any temporary internet connection issue at the client or along the internet route.
  5. In case of "Backup not changed" warning status which occurs if the client has not interchanged the USB before 12 noon on that day, no action is required from the support staff and a downtime until 1 am next day needs to be scheduled.
    • PLEASE NOTE: Failure to schedule appropriate downtime will lead to REDUNDANT ALERTS from NAGIOS every hour.

What happens if there is no remote access to NEOSYS server based in client’s premises ?

If access to the NEOSYS server is lost then we must determine the root cause by:

  1. Checking if the server is UP and running
  2. If yes, please check internet connectivity on the server
  3. If there is connectivity, please check the router for connectivity issues


Sample Response: Please note that we have currently lost access to the NEOSYS server. The server seems to be down at the moment and it seems that neosys processes are not running on the server. Kindly check if the server is UP and running. If yes, please check internet connectivity on the server. Do keep us posted on the server status so we can test connectivity from our side as well.0

New Router (Port Forwarding)

If you have changed your router then you may notice that external access to NEOSYS is unavailable.

Solution:

Setup a permanent access for NEOSYS by reconfiguring the Router / Firewall for Port Forwarding from Router to the NEOSYS Server as follows:

  1. Port 19580 > 19580 for SSH
  2. Port 4430 > 4430 for HTTPS

You can see Set Up Port Forwarding to learn how to configure your Router.

To see how to test/ troubleshoot port forwarding settings, go to Troubleshooting Port Forwarding.

Sample Response: You are requested to kindly setup a permanent access for NEOSYS by reconfiguring the Router / Firewall for Port Forwarding from Router to the NEOSYS Server,i.e. port 19580 for SSH and port 4430 for HTTPS.

Once this is complete, kindly send me an email to confirm the same so that we could test connectivity from our end as well.

Backup Procedures

Preparing daily backup report

  1. Note the success, failure and other error of the clients backup mail in an excel sheet and forward the same to your manager.
  2. If there is a backup failure or backup is not available, check wiki to take necessary steps.
  3. If there is any unknown error, forward the same to your manager.

Updating Nagios incase of failures

  1. If the backup failure is unsolved, schedule downtime Neosys service in Nagios till 01 am.
  2. If the backup did not happen because of server down. Call the IT person; ask him to re-boot the server and check wiki to do necessary step ahead and schedule downtime to Nagios for 2hours.
  3. If there is an error "Backup->Impossible" on Nagios check wiki and schedule downtime to Nagios for 2hours.

Interchange backup USB mail reminder

  1. Basically all the clients have different days to change their backup USB. All the notification can be seen on Nagios at 12.00 pm every day.
  2. When you see the change backup notification on Nagios at 12.00 pm, send a mail to the IT person or the person who changes the USB. As you send the mail schedule downtime for Neosys services for two hours.
  3. Check the Nagios at 2.00 pm if the backup USB is changed, don't send another mail and stop there. But if backup is still not yet changed, send a another mail and this time cc the mail to your manager. After sending the mail, schedule downtime for Neosys service for more two hours.
  4. Now at 4.00 pm if you still find the backup USB not changed. Call the IT person and ask him to change the USB ASAP and schedule downtime for Neosys services for one hour.
  5. Now at 5.00 pm check Nagios and see if the backup USB is changed and if not changed. Send the mail to everyone from the backup mail.
  6. If the backup is still not changed, escalate the issue to your manager.

Importance of interchanging backup USBs

If the backup USB is not interchanged on the scheduled day then the NEOSYS automated backup fails. This happens because traditionally, each USB holds backup of 7 days and using 3 different USBs we can store backups for the last 21 days enabling us to restore the system unto a time period beginning 21 days prior. If the USB is not changed then the first backup on the current USB is replaced with the new or latest backup leading to inconsistencies within the backups. Hence we must interchange the USB on schedule to avoid a backup failure the next morning.

The reasons for using multiple USBs for backup are:

  1. We can keep other USBs out of the office for safety purposes since theft or office fire/water hazards could damage the computer and the USB keys if they are all in the same place.
  2. Having multiple USBs provide safety against corrupt USBs which cannot be used to restore any backup data.

Sample Response: The NEOSYS backup has failed today as the USB was not interchanged on the scheduled day. Please interchange the USB immediately today to avoid a backup failure tomorrow morning. Also note that your scheduled day to interchange the USB next week and hence forth remains unchanged to (Scheduled Day).

Finding out which USB is inserted into the server

As we ask the client to have 3 USB's and interchange them weekly, we also need to sometimes track which one of these 3 USB's are inserted into the server. USB's can be tracked using their volume serial number in most cases. To find this out either go to the command prompt and type VOL or in the nightly backup message check for the 2nd line (which looks like this - 14/12/2009 2:45pm Media: 705B-5B5F). However serial numbers can be the same even for different USB's.

Creating and Handling passwords

Passwords made out of a pass phrase should be at least 10 characters since using initials results in a lot of i's and a's etc which reduces the effectiveness of the password and allows hacking via brute force guessing especially since windows doesnt slow down logins even if it sees thousands of password attempts.

Creating a password

Passwords are generated from a pass phrase and it is important to create a very difficult to guess pass phrase.

For example, a good pass phrase would be: Today is a good day and it is the best time to go for a holiday

The password for this would be Tiagdaiitbt2g4ah

The important instructions for the above are:

  1. You have to take the first letter of each word and that makes your password (i.e. by using initials)
  2. Wherever any word starts with a capital, then you have to take first letter as a capital (eg. For Today you will take T)
  3. Replace and with &
  4. Replace to with 2
  5. Replace for with 4

Handling passwords

  1. Never send the actual password - always send the pass phrase
  2. Make sure that the password created out of the pass phrase is at least 10 characters long since using initials results in a lot of i's and a's etc which reduces the effectiveness of the password and allows hacking via brute force guessing especially since windows doesnt slow down logins even if it sees thousands of password attempts
  3. Pass phrases are never to be sent by email, whatever the case maybe.
  4. Pass phrases can be sent by chat - however they have to be broken down in two parts and sent separately over two different messengers or if you are using Gtalk then use the 'off the record' mode.
  5. Using SMS to send pass phrases is the best known way as of now.
  6. If you save the passwords on your system in an file then:
    • Ensure that you only store pass phrases in the excel file
    • Ensure that the excel file is encrypted with a master password

NEOSYS Maintenance Window

The NEOSYS server is functional from 6am – 1am. There is a 5hr window gap for the system to perform updates & backups.

The 5hr maintenance window:-

1. At 1am – The server performs a data backup on a USB (for the respective clients) & once the backup has been completed, the system automatically generates an email addressed to the neosys staff & the respective clients.

2. At 2:45am – The main data over writes the test data on the server.

3. At 3:00am – The server by itself performs an update for Windows.

4. At 4:00am – The server performs a backup to the headquarters for clients, and then automatically generates an email addressed to the NEOSYS staff & the respective clients. This process is followed by only two clients i.e. Adline & Farouk.

5. At 6:00am – The server starts up NEOSYS.

Cutting and Pasting NEOSYS Maintenance Mode Messages into Wiki

Error messages expressed as images are not searchable.

Therefore SELECT THE TEXT OF MAINTENANCE MODE WINDOWS using right click on window heading, Edit, Mark, Copy .. not graphical copy.

Then paste the text into wiki and surround with "< pre>" and "< /pre>" tags (without the space) as follows:

Loginmessage.jpg

the result is searchable text as follows ... 

╔════════════════════════════╗
║      NEOSYS SECURITY       ║
║    What is your name ?     ║
║                            ║
║   [                    ]   ║
║                            ║
║  Please enter your name,   ║
║   or press Esc to exit.    ║
╚════════════════════════════╝

Amending/Reposting Journal Entries

In certain exceptional cases, amending/reposting of journal entries is allowed for a brief period of time to enable clients to present reports in an alternative manner. This would be subject to NEOSYS would requiring a written LETTER OF APPROVAL duly signed and stamped by the highest management of the company.

In case the client management decides to allow editing/reposting of journal entries, the following procedure is to be followed:

  1. Client must de-allocate vouchers which need to be amended
  2. NEOSYS support staff must wait for a day so that de-allocated vouchers are copied into Test database
  3. Authorise required users to amend and repost (without record) in Test database only
    ( While reposting, we have 2 options i.e. with record and without record. The 'with record' option causes the system to maintain a history of edits made. Hence, we want to repost without record so that there is no trace of the edit in the system)
  4. Amend a substantial number of vouchers in Test and verify them.
    To verify if the edits made are reflected:
  5. *Print all ledgers for the whole year
  6. *Cross-check all balances
  7. Once you verify the balances are correct in Test database, grant users permission to amend and repost in the Live database.
  8. Ask users to amend and repost vouchers in the Live database.
  9. Cross-check all balances for the current year.
  10. If you successfully verify the balances, revoke permissions immediately. Else, wait for 24 hours and revoke permissions irrespectively.

Removal of unauthorized third-party software on client servers

Rule: Any third party software that is discovered by NEOSYS support staff on client servers that has been installed without the agreement of NEOSYS should be uninstalled immediately on discovery.

However purposeful a software is, NEOSYS is contractually responsible for support and there are too many opportunities for poorly installed software to cause unpredictable damage to the NEOSYS database so NEOSYS has to have a clear and safe and simple policy to ensure the integrity of client data. Installing software without prior discussion with NEOSYS by itself indicates that insufficient care and consideration as been given to possible issues.

Any software required by client IT for some purpose may only be installed after discussion and agreement from NEOSYS support staff concerning the configuration and operation of the software.

The NEOSYS Software Licence and Support agreement requires that where NEOSYS software is installed on client servers that a dedicated server is provided and dedicated implies that no other software may be installed without the agreement of NEOSYS support.

Configuring tunnelier to autologin on opening tlp files

If you have many tunnelier tlp files in a directory and connect by opening the desired tlp file the, instead of opening the file and then clicking Login you can also right click the file and select Connect.

Alternatively, you can configure tunnelier to login (connect) automatically by following the procedure mentioned below. (Even if you configure automatic login, you can still open and not login by right clicking and choosing Open)

  1. Go to My Computer
  2. Click on Tools -> Folder Options

    Tunnauto-1.jpg

  3. Click on File Types
    • Select TLP (Bitvise Tunnelier Profile) [Type "TLP" to find it quickly]
    • Click on Advanced

      Tunnauto-2.jpg

  4. Click on Connect and Click on Set Default

    Tunnauto-3.jpg
Retrieved from "https://techwiki.neosys.com/index.php?title=Procedures&oldid=1181"