NEOSYS Technical Support Wiki - User contributions [en]

EXODUS Knowledge

2025-12-30T10:29:51Z

Gregory: /* Processes */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
General work for server services/configs. i.e /etc/nagios, /etc/apache2, /var/log/mail.log or general networking.

1: /root/exodus/
Exodus core is installed. Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus library source code which emulate the AREV system e.g var.cpp

3: /root/exodus/cli/src
Exodus Command Line Interface E.g edir, edic, list, listfiles, delete, createdb, deletedb.

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
The www/3/exodus has generic .htm, php and .js which provide basic website functionality i.e Login page, Support Menu, User details, Authorization.
NEOSYS www files linked into ./www, from /root/neosys/web, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
The dat/ has generic dictionary files. (See Section Dictionaries & Psql Functions)
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation & management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
/root/neosys has the neosys .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)

==Dictionaries & Psql Functions==

Dictionaries - data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) Field for raw data E.g "Office 44 Building Rush"
(S) Symbolic or calculated E.g for total ans=field1+field2
(G) General or Group E.g "@CRT" which contains always contains the default columns to show on the output of the list program; if no columns provided in list command.
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Delete Database dictionary file===

cli syncdat deletes database files if the dat dir is empty or contains a file 'SYNCDAT_DELETE'

<pre>
touch neosys/src/dat/dict.users/SYNC_DELETE
neosys/src~: syncdat
</pre>

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file (i.e dict.file record)=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

=====Delete dat record =====

Simply open the corresponding dat file, remove all lines and save.

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/live/lib.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
dbdelete <dbcode> #exodus cli program
OR
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging EXODUS var type variables===

The Exodus var mimics JS var in that you can assign it multiple types.

var v1 = "hello";
var v2 = 999
var v3 = 9.99;

<pre>
(gdb) print v1
$1 = {var_str = "hello", var_int = 0, var_dbl = 0, var_typ = {flags_ = 1}}

(gdb) print v2
$2 = {var_str = "", var_int = 999, var_dbl = 0, var_typ = {flags_ = 2}}

(gdb) print v3
$3 = {var_str = "", var_int = 0, var_dbl = 9.99, var_typ = {flags_ = 3}}
</pre>

=====Internals of a var explained:=====

var_str is type std::string

var_int is type int

var_dbl is type long double

var_type is type int (the var type flag)

Type flag explained:

The first 3 bits of var_typ (flags_) are used to signify which of the other variables are assigned.
Examples:

000 (binary) = 0 (flags_) = none are assigned.

100 (binary) = 1 (flags_) = var_str is assigned. (others are not)

010 (binary) = 2 (flags_) = var_int is assigned. (ditto)

001 (binary) = 4 (flags_) = var_dbl is assigned. (ditto)

Combinations:

110 (binary) = 3 (flags_) = var_str & var_int are assigned. (var_dbl is not)

v1 = "10";
(gdb) p v1
$1 = {var_str = "10", var_int = "", var_dbl = "", var_typ = {flags_ = 1}}

v1 = v1 + 0;
(gdb) n #execute that code
(gdb) p v1
$1 = {var_str = "10", var_int = "10", var_dbl = "", var_typ = {flags_ = 2}}

What happened during 'v1 = v1 + 0;':
1. v1's flag was 1, i.e only var_str was assigned.
2. Numeric operator (+) invoked
3. Attempt string to numerical type conversion either int or double.
4. Yes var_str = "10" -> var_int = 10.
5. Update flag to 2 (both var_str/int now assigned)
6. var_int = 10, + 0 = 10;

====To inspect a "common" variable (i.e those found in e.g ba_common.h)====

e.g. a variable called ba.ledgern

p ba.ledgern

OLD METHOD:
p ((subExodusProgram::ba_common)ba).ledgern

====To dump all variables in a common====

e.g all variables in ba

p (subExodusProgram::ba_common)ba

====What types are available in a program?====

<pre>info types common
info types subExodus
info types libexodus</pre>

====Other commands====

explore subExodusProgram::ba_common

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

READ: remember there is no test version of libexodus programs. This means that if libexodus programs are compiled and installed, neosys/src and exodus/services LIVE programs MUST BE UPGRADED.

#Upgrade test system to test
##If a dev system like de2, git stash any uncommitted changes other staff may be working on.
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live" in ~/neosys (to upgrade exodus/service and neosys/src live programs)
##Begin testing
#Upgrade clients
##*Client hosted - Schedule a planned upgrade with client.
##*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
##Day of upgrade, check USB backup was successful & make a container snapshot, otherwise do not proceed.
##Stop services
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live".
##Check services started automatically without issues.
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For PTCY,

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2025-12-12T08:23:27Z

Gregory: /* Debugging EXODUS var type variables */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
General work for server services/configs. i.e /etc/nagios, /etc/apache2, /var/log/mail.log or general networking.

1: /root/exodus/
Exodus core is installed. Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus library source code which emulate the AREV system e.g var.cpp

3: /root/exodus/cli/src
Exodus Command Line Interface E.g edir, edic, list, listfiles, delete, createdb, deletedb.

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
The www/3/exodus has generic .htm, php and .js which provide basic website functionality i.e Login page, Support Menu, User details, Authorization.
NEOSYS www files linked into ./www, from /root/neosys/web, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
The dat/ has generic dictionary files. (See Section Dictionaries & Psql Functions)
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation & management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
/root/neosys has the neosys .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)

==Dictionaries & Psql Functions==

Dictionaries - data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) Field for raw data E.g "Office 44 Building Rush"
(S) Symbolic or calculated E.g for total ans=field1+field2
(G) General or Group E.g "@CRT" which contains always contains the default columns to show on the output of the list program; if no columns provided in list command.
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Delete Database dictionary file===

cli syncdat deletes database files if the dat dir is empty or contains a file 'SYNCDAT_DELETE'

<pre>
touch neosys/src/dat/dict.users/SYNC_DELETE
neosys/src~: syncdat
</pre>

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file (i.e dict.file record)=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

=====Delete dat record =====

Simply open the corresponding dat file, remove all lines and save.

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
dbdelete <dbcode> #exodus cli program
OR
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging EXODUS var type variables===

The Exodus var mimics JS var in that you can assign it multiple types.

var v1 = "hello";
var v2 = 999
var v3 = 9.99;

<pre>
(gdb) print v1
$1 = {var_str = "hello", var_int = 0, var_dbl = 0, var_typ = {flags_ = 1}}

(gdb) print v2
$2 = {var_str = "", var_int = 999, var_dbl = 0, var_typ = {flags_ = 2}}

(gdb) print v3
$3 = {var_str = "", var_int = 0, var_dbl = 9.99, var_typ = {flags_ = 3}}
</pre>

=====Internals of a var explained:=====

var_str is type std::string

var_int is type int

var_dbl is type long double

var_type is type int (the var type flag)

Type flag explained:

The first 3 bits of var_typ (flags_) are used to signify which of the other variables are assigned.
Examples:

000 (binary) = 0 (flags_) = none are assigned.

100 (binary) = 1 (flags_) = var_str is assigned. (others are not)

010 (binary) = 2 (flags_) = var_int is assigned. (ditto)

001 (binary) = 4 (flags_) = var_dbl is assigned. (ditto)

Combinations:

110 (binary) = 3 (flags_) = var_str & var_int are assigned. (var_dbl is not)

v1 = "10";
(gdb) p v1
$1 = {var_str = "10", var_int = "", var_dbl = "", var_typ = {flags_ = 1}}

v1 = v1 + 0;
(gdb) n #execute that code
(gdb) p v1
$1 = {var_str = "10", var_int = "10", var_dbl = "", var_typ = {flags_ = 2}}

What happened during 'v1 = v1 + 0;':
1. v1's flag was 1, i.e only var_str was assigned.
2. Numeric operator (+) invoked
3. Attempt string to numerical type conversion either int or double.
4. Yes var_str = "10" -> var_int = 10.
5. Update flag to 2 (both var_str/int now assigned)
6. var_int = 10, + 0 = 10;

====To inspect a "common" variable (i.e those found in e.g ba_common.h)====

e.g. a variable called ba.ledgern

p ba.ledgern

OLD METHOD:
p ((subExodusProgram::ba_common)ba).ledgern

====To dump all variables in a common====

e.g all variables in ba

p (subExodusProgram::ba_common)ba

====What types are available in a program?====

<pre>info types common
info types subExodus
info types libexodus</pre>

====Other commands====

explore subExodusProgram::ba_common

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

READ: remember there is no test version of libexodus programs. This means that if libexodus programs are compiled and installed, neosys/src and exodus/services LIVE programs MUST BE UPGRADED.

#Upgrade test system to test
##If a dev system like de2, git stash any uncommitted changes other staff may be working on.
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live" in ~/neosys (to upgrade exodus/service and neosys/src live programs)
##Begin testing
#Upgrade clients
##*Client hosted - Schedule a planned upgrade with client.
##*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
##Day of upgrade, check USB backup was successful & make a container snapshot, otherwise do not proceed.
##Stop services
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live".
##Check services started automatically without issues.
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For PTCY,

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2025-12-12T08:17:34Z

Gregory: /* Postgres */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
General work for server services/configs. i.e /etc/nagios, /etc/apache2, /var/log/mail.log or general networking.

1: /root/exodus/
Exodus core is installed. Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus library source code which emulate the AREV system e.g var.cpp

3: /root/exodus/cli/src
Exodus Command Line Interface E.g edir, edic, list, listfiles, delete, createdb, deletedb.

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
The www/3/exodus has generic .htm, php and .js which provide basic website functionality i.e Login page, Support Menu, User details, Authorization.
NEOSYS www files linked into ./www, from /root/neosys/web, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
The dat/ has generic dictionary files. (See Section Dictionaries & Psql Functions)
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation & management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
/root/neosys has the neosys .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)

==Dictionaries & Psql Functions==

Dictionaries - data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) Field for raw data E.g "Office 44 Building Rush"
(S) Symbolic or calculated E.g for total ans=field1+field2
(G) General or Group E.g "@CRT" which contains always contains the default columns to show on the output of the list program; if no columns provided in list command.
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Delete Database dictionary file===

cli syncdat deletes database files if the dat dir is empty or contains a file 'SYNCDAT_DELETE'

<pre>
touch neosys/src/dat/dict.users/SYNC_DELETE
neosys/src~: syncdat
</pre>

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file (i.e dict.file record)=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

=====Delete dat record =====

Simply open the corresponding dat file, remove all lines and save.

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
dbdelete <dbcode> #exodus cli program
OR
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging EXODUS var type variables===

First quick intro of Exodus var type

var v1 = "hello";
var v2 = 999
var v3 = 9.99;

<pre>
(gdb) print v1
$1 = {var_str = "hello", var_int = 0, var_dbl = 0, var_typ = {flags_ = 1}}

(gdb) print v2
$2 = {var_str = "", var_int = 999, var_dbl = 0, var_typ = {flags_ = 2}}

(gdb) print v3
$3 = {var_str = "", var_int = 0, var_dbl = 9.99, var_typ = {flags_ = 3}}

</pre>

The Exodus var mimics JS var in that you can assign it multiple types.

var_str is type std::string
var_int is type int
var_dbl is type long double

var_type is type int (the var type flag)

===== Type flag explained: =====
The first 3 bits of var_typ (flags_) are used to signify which of the other variables are assigned.
Examples:

000 (binary) = 0 (flags_) = none are assigned.

100 (binary) = 1 (flags_) = var_str is assigned. (others are not)

010 (binary) = 2 (flags_) = var_int is assigned. (ditto)

001 (binary) = 4 (flags_) = var_dbl is assigned. (ditto)

Combinations:

110 (binary) = 3 (flags_) = var_str & var_int are assigned. (var_dbl is not)

v1 = "10";
(gdb) p v1
$1 = {var_str = "10", var_int = "", var_dbl = "", var_typ = {flags_ = 1}}

v1 = v1 + 0;
(gdb) n #execute that code
(gdb) p v1
$1 = {var_str = "10", var_int = "10", var_dbl = "", var_typ = {flags_ = 2}}

What happened during 'v1 = v1 + 0;':
1. v1's flag was 1, i.e only var_str was assigned.
2. Numeric operator (+) invoked
3. Attempt string to numerical type conversion either int or double.
4. Yes var_str = "10" -> var_int = 10.
5. Update flag to 2 (both var_str/int now assigned)
6. var_int = 10, + 0 = 10;

====To inspect a "common" variable (i.e those found in e.g ba_common.h)====

e.g. a variable called ba.ledgern

p ba.ledgern

OLD METHOD:
p ((subExodusProgram::ba_common)ba).ledgern

====To dump all variables in a common====

e.g all variables in ba

p (subExodusProgram::ba_common)ba

====What types are available in a program?====

<pre>info types common
info types subExodus
info types libexodus</pre>

====Other commands====

explore subExodusProgram::ba_common

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

READ: remember there is no test version of libexodus programs. This means that if libexodus programs are compiled and installed, neosys/src and exodus/services LIVE programs MUST BE UPGRADED.

#Upgrade test system to test
##If a dev system like de2, git stash any uncommitted changes other staff may be working on.
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live" in ~/neosys (to upgrade exodus/service and neosys/src live programs)
##Begin testing
#Upgrade clients
##*Client hosted - Schedule a planned upgrade with client.
##*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
##Day of upgrade, check USB backup was successful & make a container snapshot, otherwise do not proceed.
##Stop services
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live".
##Check services started automatically without issues.
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For PTCY,

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2025-12-12T08:14:02Z

Gregory: /* Debugging var values */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
General work for server services/configs. i.e /etc/nagios, /etc/apache2, /var/log/mail.log or general networking.

1: /root/exodus/
Exodus core is installed. Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus library source code which emulate the AREV system e.g var.cpp

3: /root/exodus/cli/src
Exodus Command Line Interface E.g edir, edic, list, listfiles, delete, createdb, deletedb.

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
The www/3/exodus has generic .htm, php and .js which provide basic website functionality i.e Login page, Support Menu, User details, Authorization.
NEOSYS www files linked into ./www, from /root/neosys/web, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
The dat/ has generic dictionary files. (See Section Dictionaries & Psql Functions)
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation & management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
/root/neosys has the neosys .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)

==Dictionaries & Psql Functions==

Dictionaries - data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) Field for raw data E.g "Office 44 Building Rush"
(S) Symbolic or calculated E.g for total ans=field1+field2
(G) General or Group E.g "@CRT" which contains always contains the default columns to show on the output of the list program; if no columns provided in list command.
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Delete Database dictionary file===

cli syncdat deletes database files if the dat dir is empty or contains a file 'SYNCDAT_DELETE'

<pre>
touch neosys/src/dat/dict.users/SYNC_DELETE
neosys/src~: syncdat
</pre>

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file (i.e dict.file record)=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

=====Delete dat record =====

Simply open the corresponding dat file, remove all lines and save.

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging EXODUS var type variables===

First quick intro of Exodus var type

var v1 = "hello";
var v2 = 999
var v3 = 9.99;

<pre>
(gdb) print v1
$1 = {var_str = "hello", var_int = 0, var_dbl = 0, var_typ = {flags_ = 1}}

(gdb) print v2
$2 = {var_str = "", var_int = 999, var_dbl = 0, var_typ = {flags_ = 2}}

(gdb) print v3
$3 = {var_str = "", var_int = 0, var_dbl = 9.99, var_typ = {flags_ = 3}}

</pre>

The Exodus var mimics JS var in that you can assign it multiple types.

var_str is type std::string
var_int is type int
var_dbl is type long double

var_type is type int (the var type flag)

===== Type flag explained: =====
The first 3 bits of var_typ (flags_) are used to signify which of the other variables are assigned.
Examples:

000 (binary) = 0 (flags_) = none are assigned.

100 (binary) = 1 (flags_) = var_str is assigned. (others are not)

010 (binary) = 2 (flags_) = var_int is assigned. (ditto)

001 (binary) = 4 (flags_) = var_dbl is assigned. (ditto)

Combinations:

110 (binary) = 3 (flags_) = var_str & var_int are assigned. (var_dbl is not)

v1 = "10";
(gdb) p v1
$1 = {var_str = "10", var_int = "", var_dbl = "", var_typ = {flags_ = 1}}

v1 = v1 + 0;
(gdb) n #execute that code
(gdb) p v1
$1 = {var_str = "10", var_int = "10", var_dbl = "", var_typ = {flags_ = 2}}

What happened during 'v1 = v1 + 0;':
1. v1's flag was 1, i.e only var_str was assigned.
2. Numeric operator (+) invoked
3. Attempt string to numerical type conversion either int or double.
4. Yes var_str = "10" -> var_int = 10.
5. Update flag to 2 (both var_str/int now assigned)
6. var_int = 10, + 0 = 10;

====To inspect a "common" variable (i.e those found in e.g ba_common.h)====

e.g. a variable called ba.ledgern

p ba.ledgern

OLD METHOD:
p ((subExodusProgram::ba_common)ba).ledgern

====To dump all variables in a common====

e.g all variables in ba

p (subExodusProgram::ba_common)ba

====What types are available in a program?====

<pre>info types common
info types subExodus
info types libexodus</pre>

====Other commands====

explore subExodusProgram::ba_common

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

READ: remember there is no test version of libexodus programs. This means that if libexodus programs are compiled and installed, neosys/src and exodus/services LIVE programs MUST BE UPGRADED.

#Upgrade test system to test
##If a dev system like de2, git stash any uncommitted changes other staff may be working on.
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live" in ~/neosys (to upgrade exodus/service and neosys/src live programs)
##Begin testing
#Upgrade clients
##*Client hosted - Schedule a planned upgrade with client.
##*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
##Day of upgrade, check USB backup was successful & make a container snapshot, otherwise do not proceed.
##Stop services
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live".
##Check services started automatically without issues.
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For PTCY,

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2025-12-12T07:45:01Z

Gregory: /* To inspect a “common” variable */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
General work for server services/configs. i.e /etc/nagios, /etc/apache2, /var/log/mail.log or general networking.

1: /root/exodus/
Exodus core is installed. Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus library source code which emulate the AREV system e.g var.cpp

3: /root/exodus/cli/src
Exodus Command Line Interface E.g edir, edic, list, listfiles, delete, createdb, deletedb.

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
The www/3/exodus has generic .htm, php and .js which provide basic website functionality i.e Login page, Support Menu, User details, Authorization.
NEOSYS www files linked into ./www, from /root/neosys/web, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
The dat/ has generic dictionary files. (See Section Dictionaries & Psql Functions)
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation & management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
/root/neosys has the neosys .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)

==Dictionaries & Psql Functions==

Dictionaries - data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) Field for raw data E.g "Office 44 Building Rush"
(S) Symbolic or calculated E.g for total ans=field1+field2
(G) General or Group E.g "@CRT" which contains always contains the default columns to show on the output of the list program; if no columns provided in list command.
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Delete Database dictionary file===

cli syncdat deletes database files if the dat dir is empty or contains a file 'SYNCDAT_DELETE'

<pre>
touch neosys/src/dat/dict.users/SYNC_DELETE
neosys/src~: syncdat
</pre>

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file (i.e dict.file record)=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

=====Delete dat record =====

Simply open the corresponding dat file, remove all lines and save.

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 4 variables of type std::string (var_str), int (var_int), long double (var_dbl) and another int (var_typ i.e flags).

The first 3 bits of var_typ (flags_) are used to signify which of the other variables are assigned.

First bit = string is un/assigned. Second bit = int is un/assigned. Third bit = double un/assigned (a combination of these bits describes whether var_str/ var_int/ var_dbl are assigned.

Examples:

000 (binary) = 0 (decimal) = non are assigned.

100 (binary) = 1 (decimal) = var_str is assigned. (var_int & var_dbl are not)

010 (binary) = 2 (decimal) = var_int is assigned. (var_str & var_dbl are not)

001 (binary) = 4 (decimal) = var_dbl is assigned. (var_str var_int are not)

Combinations:

110 (binary) = 3 (decimal) = var_str & var_int are assigned. (var_dbl is not)

====To inspect a “common” variable====

e.g. a variable called ba.ledgern

p ba.ledgern

OLD METHOD:

p ((subExodusProgram::ba_common)ba).ledgern

====To inspect a common block====

e.g. a common block ba

ptype (subExodusProgram::ba_common)ba

====To dump all variables in a common====

e.g all variables in ba

p (subExodusProgram::ba_common)ba

====What types are available in a program?====

<pre>info types common
info types subExodus
info types libexodus</pre>

====Other commands====

explore subExodusProgram::ba_common

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

READ: remember there is no test version of libexodus programs. This means that if libexodus programs are compiled and installed, neosys/src and exodus/services LIVE programs MUST BE UPGRADED.

#Upgrade test system to test
##If a dev system like de2, git stash any uncommitted changes other staff may be working on.
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live" in ~/neosys (to upgrade exodus/service and neosys/src live programs)
##Begin testing
#Upgrade clients
##*Client hosted - Schedule a planned upgrade with client.
##*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
##Day of upgrade, check USB backup was successful & make a container snapshot, otherwise do not proceed.
##Stop services
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live".
##Check services started automatically without issues.
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For PTCY,

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

New Employee Training Checklist

2025-08-06T07:48:34Z

Gregory: /* Day 9 */

==Day 1==
#Go through [[General_Office_Procedures|General Office Procedures]]
#Create a gmail account for your NEOSYS interactions e.g request for leave. Example firstname.neosys@gmail.com. Support Manager will share relevant google drive folders.
#To install and configure various software on Linux platform, see [http://itwiki.neosys.com/index.php/Setting_up_Ubuntu_on_NEOSYS_Support_Computers Setting up Ubuntu on NEOSYS Computers]
#[http://userwiki.neosys.com/index.php/Reset_Browser Reset browsers] and [http://userwiki.neosys.com/index.php/Disabling_Popup_Blocker disable pop-up blockers]
# Create an excel password file. The file name should be generic. Save with a password and share the password with your manager. Please refer link to create an appropriate password for the file ([[Procedures#Creating_and_Handling_passwords| Refer Handling passwords]]). The file should be placed under Support Staff folder in Nextcloud.
#Create signature in Thunderbird for all necessary accounts
#Mails are sent out in Calibri font with font size 11
#Create a file to store client info ( clients details) column names:- host name, company name, location of company, time zone, which server is the client's NEOSYS hosted on
#Configure your browsers according to [[Procedures#Configuring_Browsers_to_show_Javascript_errors_in_NEOSYS| Configuring Browsers to show Javascript errors]] and set it.neosys.com as Firefox home page
#Introduce nagios IT monitor
#Get NEOSYS IT to update Nagios monitoring workstations.cfg file and configure port-forwarding so that nagios can connect to the new employee's workstation
#Get a run down of client backup emails and updating backup spreadsheets.
#Create wiki ID and get NEOSYS IT to allow new users to receive wiki update emails.

==Day 2 ==
#Check the clients file
#Explain backup procedure
#Show backup emails
#Share backup files
#Make the candidate do the morning backup-check routine.
#Explain different backup failures
#Train how to investigate a backup failure. Using backup email/nagios.
#Show template emails in Thunderbird and Wiki. Focus on backup email templates.
#Go through wiki [[Procedures]]
#Introduction to NEOSYS frontend e.g login page

==Day 3==
#Do backups for the day and update the backup spreadsheet
#Train on how to schedule downtime on nagios.
#Go through the various errors in Troubleshooting nagios http://techwiki.neosys.com/index.php/Handling_Nagios_Client_Monitoring_System
#Read IT wiki "Documenting NEOSYS systems"

==Day 4==
#Handle possible backup and nagios issues of the morning/day and email clients if they need to be informed about those issues.
#Read [https://userwiki.neosys.com/index.php/Using_NEOSYS_Generally#Getting_started_with_NEOSYS| Getting started with NEOSYS]
#Explain Media Schedule (Client & Brand file, Supplier file, Vehicle file, Specification, material, dates, free ads, booking, certifying, invoicing)

==Day 5==
#Practice Media module
#Creating various Client & Brand, Vehicle and Supplier files
#Test knowledge so far on using Schedules. Manager to create two assignments of booking, cancellation, rebooking, certify & invoicing.

==Day 6==
#Quiz on Procedures, Nagios and Handling damaged files

==Day 7==
#Explain Jobs Module (Job file, Estimate file, Purchase Order file, Job Type file)
#Practice Jobs Module
#Explain Authorisation file (locks and keys, user ID, various levels etc)

==Day 8==
#Create simulated client server to learn about common scripts e.t
#Upgrade a client (how to handle patches before upgrade e.t)
#Explain DNS Zoneedit

==Day 9==
#Replicating client issues
#Backup and Restoring NEOSYS
#Introduce nextcloud/IT procedures e.g migration principles
#System configuration file features and testing each field

==Day 10==
#Request for a quick demo from the new staff

New Employee Training Checklist

2025-08-06T07:45:49Z

Gregory: /* Day 8 */

==Day 1==
#Go through [[General_Office_Procedures|General Office Procedures]]
#Create a gmail account for your NEOSYS interactions e.g request for leave. Example firstname.neosys@gmail.com. Support Manager will share relevant google drive folders.
#To install and configure various software on Linux platform, see [http://itwiki.neosys.com/index.php/Setting_up_Ubuntu_on_NEOSYS_Support_Computers Setting up Ubuntu on NEOSYS Computers]
#[http://userwiki.neosys.com/index.php/Reset_Browser Reset browsers] and [http://userwiki.neosys.com/index.php/Disabling_Popup_Blocker disable pop-up blockers]
# Create an excel password file. The file name should be generic. Save with a password and share the password with your manager. Please refer link to create an appropriate password for the file ([[Procedures#Creating_and_Handling_passwords| Refer Handling passwords]]). The file should be placed under Support Staff folder in Nextcloud.
#Create signature in Thunderbird for all necessary accounts
#Mails are sent out in Calibri font with font size 11
#Create a file to store client info ( clients details) column names:- host name, company name, location of company, time zone, which server is the client's NEOSYS hosted on
#Configure your browsers according to [[Procedures#Configuring_Browsers_to_show_Javascript_errors_in_NEOSYS| Configuring Browsers to show Javascript errors]] and set it.neosys.com as Firefox home page
#Introduce nagios IT monitor
#Get NEOSYS IT to update Nagios monitoring workstations.cfg file and configure port-forwarding so that nagios can connect to the new employee's workstation
#Get a run down of client backup emails and updating backup spreadsheets.
#Create wiki ID and get NEOSYS IT to allow new users to receive wiki update emails.

==Day 2 ==
#Check the clients file
#Explain backup procedure
#Show backup emails
#Share backup files
#Make the candidate do the morning backup-check routine.
#Explain different backup failures
#Train how to investigate a backup failure. Using backup email/nagios.
#Show template emails in Thunderbird and Wiki. Focus on backup email templates.
#Go through wiki [[Procedures]]
#Introduction to NEOSYS frontend e.g login page

==Day 3==
#Do backups for the day and update the backup spreadsheet
#Train on how to schedule downtime on nagios.
#Go through the various errors in Troubleshooting nagios http://techwiki.neosys.com/index.php/Handling_Nagios_Client_Monitoring_System
#Read IT wiki "Documenting NEOSYS systems"

==Day 4==
#Handle possible backup and nagios issues of the morning/day and email clients if they need to be informed about those issues.
#Read [https://userwiki.neosys.com/index.php/Using_NEOSYS_Generally#Getting_started_with_NEOSYS| Getting started with NEOSYS]
#Explain Media Schedule (Client & Brand file, Supplier file, Vehicle file, Specification, material, dates, free ads, booking, certifying, invoicing)

==Day 5==
#Practice Media module
#Creating various Client & Brand, Vehicle and Supplier files
#Test knowledge so far on using Schedules. Manager to create two assignments of booking, cancellation, rebooking, certify & invoicing.

==Day 6==
#Quiz on Procedures, Nagios and Handling damaged files

==Day 7==
#Explain Jobs Module (Job file, Estimate file, Purchase Order file, Job Type file)
#Practice Jobs Module
#Explain Authorisation file (locks and keys, user ID, various levels etc)

==Day 8==
#Create simulated client server to learn about common scripts e.t
#Upgrade a client (how to handle patches before upgrade e.t)
#Explain DNS Zoneedit

==Day 9==
#Replicating client issues
#Backup and Restoring NEOSYS
#Moving NEOSYS to new server/location etc
#Consolidated Backup (Autologin.sh)
#System configuration file features and testing each field

==Day 10==
#Request for a quick demo from the new staff

New Employee Training Checklist

2025-08-06T07:31:07Z

Gregory: /* Day 6 */

==Day 1==
#Go through [[General_Office_Procedures|General Office Procedures]]
#Create a gmail account for your NEOSYS interactions e.g request for leave. Example firstname.neosys@gmail.com. Support Manager will share relevant google drive folders.
#To install and configure various software on Linux platform, see [http://itwiki.neosys.com/index.php/Setting_up_Ubuntu_on_NEOSYS_Support_Computers Setting up Ubuntu on NEOSYS Computers]
#[http://userwiki.neosys.com/index.php/Reset_Browser Reset browsers] and [http://userwiki.neosys.com/index.php/Disabling_Popup_Blocker disable pop-up blockers]
# Create an excel password file. The file name should be generic. Save with a password and share the password with your manager. Please refer link to create an appropriate password for the file ([[Procedures#Creating_and_Handling_passwords| Refer Handling passwords]]). The file should be placed under Support Staff folder in Nextcloud.
#Create signature in Thunderbird for all necessary accounts
#Mails are sent out in Calibri font with font size 11
#Create a file to store client info ( clients details) column names:- host name, company name, location of company, time zone, which server is the client's NEOSYS hosted on
#Configure your browsers according to [[Procedures#Configuring_Browsers_to_show_Javascript_errors_in_NEOSYS| Configuring Browsers to show Javascript errors]] and set it.neosys.com as Firefox home page
#Introduce nagios IT monitor
#Get NEOSYS IT to update Nagios monitoring workstations.cfg file and configure port-forwarding so that nagios can connect to the new employee's workstation
#Get a run down of client backup emails and updating backup spreadsheets.
#Create wiki ID and get NEOSYS IT to allow new users to receive wiki update emails.

==Day 2 ==
#Check the clients file
#Explain backup procedure
#Show backup emails
#Share backup files
#Make the candidate do the morning backup-check routine.
#Explain different backup failures
#Train how to investigate a backup failure. Using backup email/nagios.
#Show template emails in Thunderbird and Wiki. Focus on backup email templates.
#Go through wiki [[Procedures]]
#Introduction to NEOSYS frontend e.g login page

==Day 3==
#Do backups for the day and update the backup spreadsheet
#Train on how to schedule downtime on nagios.
#Go through the various errors in Troubleshooting nagios http://techwiki.neosys.com/index.php/Handling_Nagios_Client_Monitoring_System
#Read IT wiki "Documenting NEOSYS systems"

==Day 4==
#Handle possible backup and nagios issues of the morning/day and email clients if they need to be informed about those issues.
#Read [https://userwiki.neosys.com/index.php/Using_NEOSYS_Generally#Getting_started_with_NEOSYS| Getting started with NEOSYS]
#Explain Media Schedule (Client & Brand file, Supplier file, Vehicle file, Specification, material, dates, free ads, booking, certifying, invoicing)

==Day 5==
#Practice Media module
#Creating various Client & Brand, Vehicle and Supplier files
#Test knowledge so far on using Schedules. Manager to create two assignments of booking, cancellation, rebooking, certify & invoicing.

==Day 6==
#Quiz on Procedures, Nagios and Handling damaged files

==Day 7==
#Jobs Module completely and practice
#Explain Authorisation file (locks and keys, user ID, various levels etc)

==Day 8==
#How to Upgrade NEOSYS
#Upgrade a client (use TEST installation)
#Explain Zone edit, DNS

==Day 9==
#Replicating client issues
#Backup and Restoring NEOSYS
#Moving NEOSYS to new server/location etc
#Consolidated Backup (Autologin.sh)
#System configuration file features and testing each field

==Day 10==
#Request for a quick demo from the new staff

New Employee Training Checklist

2025-08-06T07:30:31Z

Gregory: /* Day 5 */

==Day 1==
#Go through [[General_Office_Procedures|General Office Procedures]]
#Create a gmail account for your NEOSYS interactions e.g request for leave. Example firstname.neosys@gmail.com. Support Manager will share relevant google drive folders.
#To install and configure various software on Linux platform, see [http://itwiki.neosys.com/index.php/Setting_up_Ubuntu_on_NEOSYS_Support_Computers Setting up Ubuntu on NEOSYS Computers]
#[http://userwiki.neosys.com/index.php/Reset_Browser Reset browsers] and [http://userwiki.neosys.com/index.php/Disabling_Popup_Blocker disable pop-up blockers]
# Create an excel password file. The file name should be generic. Save with a password and share the password with your manager. Please refer link to create an appropriate password for the file ([[Procedures#Creating_and_Handling_passwords| Refer Handling passwords]]). The file should be placed under Support Staff folder in Nextcloud.
#Create signature in Thunderbird for all necessary accounts
#Mails are sent out in Calibri font with font size 11
#Create a file to store client info ( clients details) column names:- host name, company name, location of company, time zone, which server is the client's NEOSYS hosted on
#Configure your browsers according to [[Procedures#Configuring_Browsers_to_show_Javascript_errors_in_NEOSYS| Configuring Browsers to show Javascript errors]] and set it.neosys.com as Firefox home page
#Introduce nagios IT monitor
#Get NEOSYS IT to update Nagios monitoring workstations.cfg file and configure port-forwarding so that nagios can connect to the new employee's workstation
#Get a run down of client backup emails and updating backup spreadsheets.
#Create wiki ID and get NEOSYS IT to allow new users to receive wiki update emails.

==Day 2 ==
#Check the clients file
#Explain backup procedure
#Show backup emails
#Share backup files
#Make the candidate do the morning backup-check routine.
#Explain different backup failures
#Train how to investigate a backup failure. Using backup email/nagios.
#Show template emails in Thunderbird and Wiki. Focus on backup email templates.
#Go through wiki [[Procedures]]
#Introduction to NEOSYS frontend e.g login page

==Day 3==
#Do backups for the day and update the backup spreadsheet
#Train on how to schedule downtime on nagios.
#Go through the various errors in Troubleshooting nagios http://techwiki.neosys.com/index.php/Handling_Nagios_Client_Monitoring_System
#Read IT wiki "Documenting NEOSYS systems"

==Day 4==
#Handle possible backup and nagios issues of the morning/day and email clients if they need to be informed about those issues.
#Read [https://userwiki.neosys.com/index.php/Using_NEOSYS_Generally#Getting_started_with_NEOSYS| Getting started with NEOSYS]
#Explain Media Schedule (Client & Brand file, Supplier file, Vehicle file, Specification, material, dates, free ads, booking, certifying, invoicing)

==Day 5==
#Practice Media module
#Creating various Client & Brand, Vehicle and Supplier files
#Test knowledge so far on using Schedules. Manager to create two assignments of booking, cancellation, rebooking, certify & invoicing.

==Day 6==
#Handling damaged file with examples
#Quiz on Procedures, Nagios and Handling damaged files

==Day 7==
#Jobs Module completely and practice
#Explain Authorisation file (locks and keys, user ID, various levels etc)

==Day 8==
#How to Upgrade NEOSYS
#Upgrade a client (use TEST installation)
#Explain Zone edit, DNS

==Day 9==
#Replicating client issues
#Backup and Restoring NEOSYS
#Moving NEOSYS to new server/location etc
#Consolidated Backup (Autologin.sh)
#System configuration file features and testing each field

==Day 10==
#Request for a quick demo from the new staff

New Employee Training Checklist

2025-08-06T07:28:25Z

Gregory: /* Day 4 */

==Day 1==
#Go through [[General_Office_Procedures|General Office Procedures]]
#Create a gmail account for your NEOSYS interactions e.g request for leave. Example firstname.neosys@gmail.com. Support Manager will share relevant google drive folders.
#To install and configure various software on Linux platform, see [http://itwiki.neosys.com/index.php/Setting_up_Ubuntu_on_NEOSYS_Support_Computers Setting up Ubuntu on NEOSYS Computers]
#[http://userwiki.neosys.com/index.php/Reset_Browser Reset browsers] and [http://userwiki.neosys.com/index.php/Disabling_Popup_Blocker disable pop-up blockers]
# Create an excel password file. The file name should be generic. Save with a password and share the password with your manager. Please refer link to create an appropriate password for the file ([[Procedures#Creating_and_Handling_passwords| Refer Handling passwords]]). The file should be placed under Support Staff folder in Nextcloud.
#Create signature in Thunderbird for all necessary accounts
#Mails are sent out in Calibri font with font size 11
#Create a file to store client info ( clients details) column names:- host name, company name, location of company, time zone, which server is the client's NEOSYS hosted on
#Configure your browsers according to [[Procedures#Configuring_Browsers_to_show_Javascript_errors_in_NEOSYS| Configuring Browsers to show Javascript errors]] and set it.neosys.com as Firefox home page
#Introduce nagios IT monitor
#Get NEOSYS IT to update Nagios monitoring workstations.cfg file and configure port-forwarding so that nagios can connect to the new employee's workstation
#Get a run down of client backup emails and updating backup spreadsheets.
#Create wiki ID and get NEOSYS IT to allow new users to receive wiki update emails.

==Day 2 ==
#Check the clients file
#Explain backup procedure
#Show backup emails
#Share backup files
#Make the candidate do the morning backup-check routine.
#Explain different backup failures
#Train how to investigate a backup failure. Using backup email/nagios.
#Show template emails in Thunderbird and Wiki. Focus on backup email templates.
#Go through wiki [[Procedures]]
#Introduction to NEOSYS frontend e.g login page

==Day 3==
#Do backups for the day and update the backup spreadsheet
#Train on how to schedule downtime on nagios.
#Go through the various errors in Troubleshooting nagios http://techwiki.neosys.com/index.php/Handling_Nagios_Client_Monitoring_System
#Read IT wiki "Documenting NEOSYS systems"

==Day 4==
#Handle possible backup and nagios issues of the morning/day and email clients if they need to be informed about those issues.
#Read [https://userwiki.neosys.com/index.php/Using_NEOSYS_Generally#Getting_started_with_NEOSYS| Getting started with NEOSYS]
#Explain Media Schedule (Client & Brand file, Supplier file, Vehicle file, Specification, material, dates, free ads, booking, certifying, invoicing)

==Day 5==
#2 Assignments on Schedules ( booking, cancelation, rebooking, certify & invoicing )
#Creating on various client and brand, vehicle and supplier files
#Practice Media module

==Day 6==
#Handling damaged file with examples
#Quiz on Procedures, Nagios and Handling damaged files

==Day 7==
#Jobs Module completely and practice
#Explain Authorisation file (locks and keys, user ID, various levels etc)

==Day 8==
#How to Upgrade NEOSYS
#Upgrade a client (use TEST installation)
#Explain Zone edit, DNS

==Day 9==
#Replicating client issues
#Backup and Restoring NEOSYS
#Moving NEOSYS to new server/location etc
#Consolidated Backup (Autologin.sh)
#System configuration file features and testing each field

==Day 10==
#Request for a quick demo from the new staff

New Employee Training Checklist

2025-08-06T07:23:49Z

Gregory: /* Day 3 */

==Day 1==
#Go through [[General_Office_Procedures|General Office Procedures]]
#Create a gmail account for your NEOSYS interactions e.g request for leave. Example firstname.neosys@gmail.com. Support Manager will share relevant google drive folders.
#To install and configure various software on Linux platform, see [http://itwiki.neosys.com/index.php/Setting_up_Ubuntu_on_NEOSYS_Support_Computers Setting up Ubuntu on NEOSYS Computers]
#[http://userwiki.neosys.com/index.php/Reset_Browser Reset browsers] and [http://userwiki.neosys.com/index.php/Disabling_Popup_Blocker disable pop-up blockers]
# Create an excel password file. The file name should be generic. Save with a password and share the password with your manager. Please refer link to create an appropriate password for the file ([[Procedures#Creating_and_Handling_passwords| Refer Handling passwords]]). The file should be placed under Support Staff folder in Nextcloud.
#Create signature in Thunderbird for all necessary accounts
#Mails are sent out in Calibri font with font size 11
#Create a file to store client info ( clients details) column names:- host name, company name, location of company, time zone, which server is the client's NEOSYS hosted on
#Configure your browsers according to [[Procedures#Configuring_Browsers_to_show_Javascript_errors_in_NEOSYS| Configuring Browsers to show Javascript errors]] and set it.neosys.com as Firefox home page
#Introduce nagios IT monitor
#Get NEOSYS IT to update Nagios monitoring workstations.cfg file and configure port-forwarding so that nagios can connect to the new employee's workstation
#Get a run down of client backup emails and updating backup spreadsheets.
#Create wiki ID and get NEOSYS IT to allow new users to receive wiki update emails.

==Day 2 ==
#Check the clients file
#Explain backup procedure
#Show backup emails
#Share backup files
#Make the candidate do the morning backup-check routine.
#Explain different backup failures
#Train how to investigate a backup failure. Using backup email/nagios.
#Show template emails in Thunderbird and Wiki. Focus on backup email templates.
#Go through wiki [[Procedures]]
#Introduction to NEOSYS frontend e.g login page

==Day 3==
#Do backups for the day and update the backup spreadsheet
#Train on how to schedule downtime on nagios.
#Go through the various errors in Troubleshooting nagios http://techwiki.neosys.com/index.php/Handling_Nagios_Client_Monitoring_System
#Read IT wiki "Documenting NEOSYS systems"

==Day 4==
#Handle backup issues, critical issues on nagios
#Email clients if they need to be informed about issues.
#Explain Getting Started and NEOSYS login process
#Explain Media Schedule (per brand, Client Brand file, Vehicle file, specification, material, dates, free ads, Supplier file, booking, certifying, invoicing)

==Day 5==
#2 Assignments on Schedules ( booking, cancelation, rebooking, certify & invoicing )
#Creating on various client and brand, vehicle and supplier files
#Practice Media module

==Day 6==
#Handling damaged file with examples
#Quiz on Procedures, Nagios and Handling damaged files

==Day 7==
#Jobs Module completely and practice
#Explain Authorisation file (locks and keys, user ID, various levels etc)

==Day 8==
#How to Upgrade NEOSYS
#Upgrade a client (use TEST installation)
#Explain Zone edit, DNS

==Day 9==
#Replicating client issues
#Backup and Restoring NEOSYS
#Moving NEOSYS to new server/location etc
#Consolidated Backup (Autologin.sh)
#System configuration file features and testing each field

==Day 10==
#Request for a quick demo from the new staff

New Employee Training Checklist

2025-08-06T07:22:48Z

Gregory: /* Day 2 */

==Day 1==
#Go through [[General_Office_Procedures|General Office Procedures]]
#Create a gmail account for your NEOSYS interactions e.g request for leave. Example firstname.neosys@gmail.com. Support Manager will share relevant google drive folders.
#To install and configure various software on Linux platform, see [http://itwiki.neosys.com/index.php/Setting_up_Ubuntu_on_NEOSYS_Support_Computers Setting up Ubuntu on NEOSYS Computers]
#[http://userwiki.neosys.com/index.php/Reset_Browser Reset browsers] and [http://userwiki.neosys.com/index.php/Disabling_Popup_Blocker disable pop-up blockers]
# Create an excel password file. The file name should be generic. Save with a password and share the password with your manager. Please refer link to create an appropriate password for the file ([[Procedures#Creating_and_Handling_passwords| Refer Handling passwords]]). The file should be placed under Support Staff folder in Nextcloud.
#Create signature in Thunderbird for all necessary accounts
#Mails are sent out in Calibri font with font size 11
#Create a file to store client info ( clients details) column names:- host name, company name, location of company, time zone, which server is the client's NEOSYS hosted on
#Configure your browsers according to [[Procedures#Configuring_Browsers_to_show_Javascript_errors_in_NEOSYS| Configuring Browsers to show Javascript errors]] and set it.neosys.com as Firefox home page
#Introduce nagios IT monitor
#Get NEOSYS IT to update Nagios monitoring workstations.cfg file and configure port-forwarding so that nagios can connect to the new employee's workstation
#Get a run down of client backup emails and updating backup spreadsheets.
#Create wiki ID and get NEOSYS IT to allow new users to receive wiki update emails.

==Day 2 ==
#Check the clients file
#Explain backup procedure
#Show backup emails
#Share backup files
#Make the candidate do the morning backup-check routine.
#Explain different backup failures
#Train how to investigate a backup failure. Using backup email/nagios.
#Show template emails in Thunderbird and Wiki. Focus on backup email templates.
#Go through wiki [[Procedures]]
#Introduction to NEOSYS frontend e.g login page

==Day 3==
#Go through templates folder in support and backups email accounts.
#Show examples on when each backup template can be sent
#Do backups for the day and update the backup spreadsheet
#Schedule downtime on nagios
#Go through the various errors in Troubleshooting nagios http://techwiki.neosys.com/index.php/Handling_Nagios_Client_Monitoring_System
#Read IT wiki "Documenting NEOSYS systems"

==Day 4==
#Handle backup issues, critical issues on nagios
#Email clients if they need to be informed about issues.
#Explain Getting Started and NEOSYS login process
#Explain Media Schedule (per brand, Client Brand file, Vehicle file, specification, material, dates, free ads, Supplier file, booking, certifying, invoicing)

==Day 5==
#2 Assignments on Schedules ( booking, cancelation, rebooking, certify & invoicing )
#Creating on various client and brand, vehicle and supplier files
#Practice Media module

==Day 6==
#Handling damaged file with examples
#Quiz on Procedures, Nagios and Handling damaged files

==Day 7==
#Jobs Module completely and practice
#Explain Authorisation file (locks and keys, user ID, various levels etc)

==Day 8==
#How to Upgrade NEOSYS
#Upgrade a client (use TEST installation)
#Explain Zone edit, DNS

==Day 9==
#Replicating client issues
#Backup and Restoring NEOSYS
#Moving NEOSYS to new server/location etc
#Consolidated Backup (Autologin.sh)
#System configuration file features and testing each field

==Day 10==
#Request for a quick demo from the new staff

New Employee Training Checklist

2025-08-06T07:22:22Z

Gregory: /* Day 2 */

==Day 1==
#Go through [[General_Office_Procedures|General Office Procedures]]
#Create a gmail account for your NEOSYS interactions e.g request for leave. Example firstname.neosys@gmail.com. Support Manager will share relevant google drive folders.
#To install and configure various software on Linux platform, see [http://itwiki.neosys.com/index.php/Setting_up_Ubuntu_on_NEOSYS_Support_Computers Setting up Ubuntu on NEOSYS Computers]
#[http://userwiki.neosys.com/index.php/Reset_Browser Reset browsers] and [http://userwiki.neosys.com/index.php/Disabling_Popup_Blocker disable pop-up blockers]
# Create an excel password file. The file name should be generic. Save with a password and share the password with your manager. Please refer link to create an appropriate password for the file ([[Procedures#Creating_and_Handling_passwords| Refer Handling passwords]]). The file should be placed under Support Staff folder in Nextcloud.
#Create signature in Thunderbird for all necessary accounts
#Mails are sent out in Calibri font with font size 11
#Create a file to store client info ( clients details) column names:- host name, company name, location of company, time zone, which server is the client's NEOSYS hosted on
#Configure your browsers according to [[Procedures#Configuring_Browsers_to_show_Javascript_errors_in_NEOSYS| Configuring Browsers to show Javascript errors]] and set it.neosys.com as Firefox home page
#Introduce nagios IT monitor
#Get NEOSYS IT to update Nagios monitoring workstations.cfg file and configure port-forwarding so that nagios can connect to the new employee's workstation
#Get a run down of client backup emails and updating backup spreadsheets.
#Create wiki ID and get NEOSYS IT to allow new users to receive wiki update emails.

==Day 2 ==
#Check the clients file
#Explain backup procedure
#Show backup emails
#Share backup files
#Make the candidate do the morning backup-check routine.
#Explain different backup failures
#Train how to investigate a backup failure. Using backup email/nagios.
#Show template emails in Thunderbird and Wiki. Focus on backups templates.
#Go through wiki [[Procedures]]
#Introduction to NEOSYS frontend e.g login page

==Day 3==
#Go through templates folder in support and backups email accounts.
#Show examples on when each backup template can be sent
#Do backups for the day and update the backup spreadsheet
#Schedule downtime on nagios
#Go through the various errors in Troubleshooting nagios http://techwiki.neosys.com/index.php/Handling_Nagios_Client_Monitoring_System
#Read IT wiki "Documenting NEOSYS systems"

==Day 4==
#Handle backup issues, critical issues on nagios
#Email clients if they need to be informed about issues.
#Explain Getting Started and NEOSYS login process
#Explain Media Schedule (per brand, Client Brand file, Vehicle file, specification, material, dates, free ads, Supplier file, booking, certifying, invoicing)

==Day 5==
#2 Assignments on Schedules ( booking, cancelation, rebooking, certify & invoicing )
#Creating on various client and brand, vehicle and supplier files
#Practice Media module

==Day 6==
#Handling damaged file with examples
#Quiz on Procedures, Nagios and Handling damaged files

==Day 7==
#Jobs Module completely and practice
#Explain Authorisation file (locks and keys, user ID, various levels etc)

==Day 8==
#How to Upgrade NEOSYS
#Upgrade a client (use TEST installation)
#Explain Zone edit, DNS

==Day 9==
#Replicating client issues
#Backup and Restoring NEOSYS
#Moving NEOSYS to new server/location etc
#Consolidated Backup (Autologin.sh)
#System configuration file features and testing each field

==Day 10==
#Request for a quick demo from the new staff

New Employee Training Checklist

2025-08-06T07:20:06Z

Gregory: /* Day 2 */

==Day 1==
#Go through [[General_Office_Procedures|General Office Procedures]]
#Create a gmail account for your NEOSYS interactions e.g request for leave. Example firstname.neosys@gmail.com. Support Manager will share relevant google drive folders.
#To install and configure various software on Linux platform, see [http://itwiki.neosys.com/index.php/Setting_up_Ubuntu_on_NEOSYS_Support_Computers Setting up Ubuntu on NEOSYS Computers]
#[http://userwiki.neosys.com/index.php/Reset_Browser Reset browsers] and [http://userwiki.neosys.com/index.php/Disabling_Popup_Blocker disable pop-up blockers]
# Create an excel password file. The file name should be generic. Save with a password and share the password with your manager. Please refer link to create an appropriate password for the file ([[Procedures#Creating_and_Handling_passwords| Refer Handling passwords]]). The file should be placed under Support Staff folder in Nextcloud.
#Create signature in Thunderbird for all necessary accounts
#Mails are sent out in Calibri font with font size 11
#Create a file to store client info ( clients details) column names:- host name, company name, location of company, time zone, which server is the client's NEOSYS hosted on
#Configure your browsers according to [[Procedures#Configuring_Browsers_to_show_Javascript_errors_in_NEOSYS| Configuring Browsers to show Javascript errors]] and set it.neosys.com as Firefox home page
#Introduce nagios IT monitor
#Get NEOSYS IT to update Nagios monitoring workstations.cfg file and configure port-forwarding so that nagios can connect to the new employee's workstation
#Get a run down of client backup emails and updating backup spreadsheets.
#Create wiki ID and get NEOSYS IT to allow new users to receive wiki update emails.

==Day 2 ==
#Check the clients file
#Explain backup procedure
#Show backup emails
#Share backup files
#Make the candidate do the morning backup-check routine.
#Explain different backup failures
#Train how to investigate a backup failure. Using backup email/nagios.
#Show template emails in Thunderbird and Wiki
#Go through wiki [[Procedures]]
#Introduction to NEOSYS frontend e.g login page

==Day 3==
#Go through templates folder in support and backups email accounts.
#Show examples on when each backup template can be sent
#Do backups for the day and update the backup spreadsheet
#Schedule downtime on nagios
#Go through the various errors in Troubleshooting nagios http://techwiki.neosys.com/index.php/Handling_Nagios_Client_Monitoring_System
#Read IT wiki "Documenting NEOSYS systems"

==Day 4==
#Handle backup issues, critical issues on nagios
#Email clients if they need to be informed about issues.
#Explain Getting Started and NEOSYS login process
#Explain Media Schedule (per brand, Client Brand file, Vehicle file, specification, material, dates, free ads, Supplier file, booking, certifying, invoicing)

==Day 5==
#2 Assignments on Schedules ( booking, cancelation, rebooking, certify & invoicing )
#Creating on various client and brand, vehicle and supplier files
#Practice Media module

==Day 6==
#Handling damaged file with examples
#Quiz on Procedures, Nagios and Handling damaged files

==Day 7==
#Jobs Module completely and practice
#Explain Authorisation file (locks and keys, user ID, various levels etc)

==Day 8==
#How to Upgrade NEOSYS
#Upgrade a client (use TEST installation)
#Explain Zone edit, DNS

==Day 9==
#Replicating client issues
#Backup and Restoring NEOSYS
#Moving NEOSYS to new server/location etc
#Consolidated Backup (Autologin.sh)
#System configuration file features and testing each field

==Day 10==
#Request for a quick demo from the new staff

New Employee Training Checklist

2025-08-06T07:13:15Z

Gregory: /* Day 1 */ Revised for 2025

==Day 1==
#Go through [[General_Office_Procedures|General Office Procedures]]
#Create a gmail account for your NEOSYS interactions e.g request for leave. Example firstname.neosys@gmail.com. Support Manager will share relevant google drive folders.
#To install and configure various software on Linux platform, see [http://itwiki.neosys.com/index.php/Setting_up_Ubuntu_on_NEOSYS_Support_Computers Setting up Ubuntu on NEOSYS Computers]
#[http://userwiki.neosys.com/index.php/Reset_Browser Reset browsers] and [http://userwiki.neosys.com/index.php/Disabling_Popup_Blocker disable pop-up blockers]
# Create an excel password file. The file name should be generic. Save with a password and share the password with your manager. Please refer link to create an appropriate password for the file ([[Procedures#Creating_and_Handling_passwords| Refer Handling passwords]]). The file should be placed under Support Staff folder in Nextcloud.
#Create signature in Thunderbird for all necessary accounts
#Mails are sent out in Calibri font with font size 11
#Create a file to store client info ( clients details) column names:- host name, company name, location of company, time zone, which server is the client's NEOSYS hosted on
#Configure your browsers according to [[Procedures#Configuring_Browsers_to_show_Javascript_errors_in_NEOSYS| Configuring Browsers to show Javascript errors]] and set it.neosys.com as Firefox home page
#Introduce nagios IT monitor
#Get NEOSYS IT to update Nagios monitoring workstations.cfg file and configure port-forwarding so that nagios can connect to the new employee's workstation
#Get a run down of client backup emails and updating backup spreadsheets.
#Create wiki ID and get NEOSYS IT to allow new users to receive wiki update emails.

==Day 2 ==
#Check the clients file
#Explain backup procedure
#Show backup emails
#Share backup files
#Make the candidate do the morning backup-check routine.
#Explain http://techwiki.neosys.com/index.php/Procedures#NEOSYS_Maintenance_Window
#Explain different backup failures
#Various checks on why and how a backup failed. (mail in inbox, nagios trends, neosys log, server event viewer)
#Show canned emails
#Go through wiki [[Procedures]]
#NEOSYS login page and configuring NEOSYS

==Day 3==
#Go through templates folder in support and backups email accounts.
#Show examples on when each backup template can be sent
#Do backups for the day and update the backup spreadsheet
#Schedule downtime on nagios
#Go through the various errors in Troubleshooting nagios http://techwiki.neosys.com/index.php/Handling_Nagios_Client_Monitoring_System
#Read IT wiki "Documenting NEOSYS systems"

==Day 4==
#Handle backup issues, critical issues on nagios
#Email clients if they need to be informed about issues.
#Explain Getting Started and NEOSYS login process
#Explain Media Schedule (per brand, Client Brand file, Vehicle file, specification, material, dates, free ads, Supplier file, booking, certifying, invoicing)

==Day 5==
#2 Assignments on Schedules ( booking, cancelation, rebooking, certify & invoicing )
#Creating on various client and brand, vehicle and supplier files
#Practice Media module

==Day 6==
#Handling damaged file with examples
#Quiz on Procedures, Nagios and Handling damaged files

==Day 7==
#Jobs Module completely and practice
#Explain Authorisation file (locks and keys, user ID, various levels etc)

==Day 8==
#How to Upgrade NEOSYS
#Upgrade a client (use TEST installation)
#Explain Zone edit, DNS

==Day 9==
#Replicating client issues
#Backup and Restoring NEOSYS
#Moving NEOSYS to new server/location etc
#Consolidated Backup (Autologin.sh)
#System configuration file features and testing each field

==Day 10==
#Request for a quick demo from the new staff

New Employee Training Checklist

2025-08-06T07:02:44Z

Gregory: /* Day 1 */

==Day 1==
#Go through [[General_Office_Procedures|General Office Procedures]]
#Create email Id eg :- firstname.neosys@gmail.com. Support Manager will share relevant google drive folders.
#To install and configure various software on Linux platform, see [http://itwiki.neosys.com/index.php/Setting_up_Ubuntu_on_NEOSYS_Support_Computers Setting up Ubuntu on NEOSYS Computers]
#[http://userwiki.neosys.com/index.php/Reset_Browser Reset browsers] and [http://userwiki.neosys.com/index.php/Disabling_Popup_Blocker disable pop-up blockers]
#Password file:- This file contains all usernames and passwords. The file name should be generic. The file is password protected, please refer link to create an appropriate password for the file ([[Procedures#Creating_and_Handling_passwords| Refer Handling passwords]]). The file should be placed under Support Staff folder in Nextcloud. The aim is to protect the file from being accessed or modified by another person.
#Create signature in Thunderbird for all necessary accounts
#Mails are sent out in Calibri font with font size 11
#Create a file to store client info ( clients details) column names:- host name, company name, location of company, time zone, which server is the client's NEOSYS hosted on
#Configure your browsers according to [[Procedures#Configuring_Browsers_to_show_Javascript_errors_in_NEOSYS| Configuring Browsers to show Javascript errors]] and set vm1.neosys.com as Firefox home page
#Introduce nagios
#Get NEOSYS IT to update Nagios monitoring workstations.cfg file and configure port-forwarding so that nagios can connect to the new employee's workstation
#Read wiki page “Backup and Restore”
#Create wiki ID and get NEOSYS IT to allow new users to receive wiki update emails
#http://network-tools.com

==Day 2 ==
#Check the clients file
#Explain backup procedure
#Show backup emails
#Share backup files
#Make the candidate do the morning backup-check routine.
#Explain http://techwiki.neosys.com/index.php/Procedures#NEOSYS_Maintenance_Window
#Explain different backup failures
#Various checks on why and how a backup failed. (mail in inbox, nagios trends, neosys log, server event viewer)
#Show canned emails
#Go through wiki [[Procedures]]
#NEOSYS login page and configuring NEOSYS

==Day 3==
#Go through templates folder in support and backups email accounts.
#Show examples on when each backup template can be sent
#Do backups for the day and update the backup spreadsheet
#Schedule downtime on nagios
#Go through the various errors in Troubleshooting nagios http://techwiki.neosys.com/index.php/Handling_Nagios_Client_Monitoring_System
#Read IT wiki "Documenting NEOSYS systems"

==Day 4==
#Handle backup issues, critical issues on nagios
#Email clients if they need to be informed about issues.
#Explain Getting Started and NEOSYS login process
#Explain Media Schedule (per brand, Client Brand file, Vehicle file, specification, material, dates, free ads, Supplier file, booking, certifying, invoicing)

==Day 5==
#2 Assignments on Schedules ( booking, cancelation, rebooking, certify & invoicing )
#Creating on various client and brand, vehicle and supplier files
#Practice Media module

==Day 6==
#Handling damaged file with examples
#Quiz on Procedures, Nagios and Handling damaged files

==Day 7==
#Jobs Module completely and practice
#Explain Authorisation file (locks and keys, user ID, various levels etc)

==Day 8==
#How to Upgrade NEOSYS
#Upgrade a client (use TEST installation)
#Explain Zone edit, DNS

==Day 9==
#Replicating client issues
#Backup and Restoring NEOSYS
#Moving NEOSYS to new server/location etc
#Consolidated Backup (Autologin.sh)
#System configuration file features and testing each field

==Day 10==
#Request for a quick demo from the new staff

Benchmarking NEOSYS

2024-12-12T05:34:56Z

Gregory: /* Benchmarking the speed of NEOSYS over the network */

=== Measuring the speed of NEOSYS on the server ===

In NEOSYS maintenance mode press F5:

FILESPEED

Measures the speed at to create and delete a file with 1000x1Kb records - lower time is better

Example speed is 1 second on a high quality notebook 7200rpm Hitachi deskstar
Speeds less than 8 seconds are acceptable for small installations
Speeds less than 4 seconds are acceptable for larger installations
Antivirus usually needs to be configured to exclude OV or OV? files. See section [[Configuring Antivirus for NEOSYS]]

PROCSPEED

Measures the processor speed – higher speed is better

Example speed is 540K tests per second on an Intel Core Duo 1.8Ghz

MEMSPEED

Measures the memory speed – higher speed is better

Example speed is 27K tests per second on an Intel Core Duo 1.8Ghz with
667MHz DDR 2 SDRAM

=== Benchmarking the speed of NEOSYS over the network ===

Test speed using Menu > Help > System Speed Test: (use option 100 for the number of times to repeat)

The three loopback tests checks the connection from:
#Web server loopback: User PC -> Apache2 and back.
#EXODUS Server loopback: User PC -> Apache2 -> NEOSYS service. (Sends request "TEST", which prompts listen program to reply 'OK')
#EXODUS application loopback: User PC -> Apache2 -> NEOSYS service -> database. (Sends a SELECT currencies command by default)

Results are to be noted in the installation check list.

=== Getting CPU characteristics ===

Intel Processor Identification Utility

http://downloadfinder.intel.com/scripts-df-external/confirm.aspx?httpDown=http://downloadmirror.intel.com/df-support/7838/eng/pidenu12.msi&agr=N&ProductID=528&DwnldId=7838&strOSs=All&OSFullName=All%20Operating%20Systems&lang=eng

Technical / Hardware requirements

2024-07-22T05:25:40Z

Gregory: /* Software requirements */

===Hardware requirements===

*Entry level physical or virtual server *dedicated to NEOSYS only* (in case of virtual machine, the host machine MUST be dedicated to NEOSYS and access given to support team, otherwise issues on the host machine will be untraceable by support team and NEOSYS will not be responsible for overall performance)
*1 x Quad core CPU (minimum)
*8 GB RAM (minimum)
*Minimum 100GB SSD Enterprise Write intensive (recommend 2x for mirroring)

===Software requirements===

*Ubuntu Server 24.04 LTS
*Configured with an internal static IP (ie dynamic IP delivered by DHCP is not allowed)
*Strictly no additional software installed

===Web Browser on User Workstations===

Firefox, Chrome or MS Edge.

Browsers must have pop-up blockers turned off and must not have any 3rd party toolbars installed, that can potentially interfere with the NEOSYS user interface.

===Backup requirements===
If entry level physical server or if virtual server that supports USB passthrough (e.g. VMWare vSphere version 4.1 and above):

*3 x 16 GB USB drives for backup
*Someone to change the USB drives on a scheduled basis (weekly)

If virtual server that does not support USB passthrough:

*Client must *at their own responsibility* agree to arrange copy of the backup files to an external backup location every day.

Also refer to [[Backup_and_Restore#Backup_in_virtual_server| Backup to virtual server]]

===Router requirements===

*Port forwarding:
**SSH: port 19581 (router) -> port 19581 (server) for Host.
**SSH: port 19582 (router) -> port 19582 (server) for lxc container.
**HTTPS: port 4431 (router) -> port 4431 (server) (Required only if NEOSYS needs to be accessed from outside office and prior management approval for this is obtained)
*Access to send email via an internal email server

===Remote support requirement===

[[Letter_to_obtain_agreement_of_client_IT_staff_to_provide_remote_access| Letter to obtain agreement of client IT staff to provide remote access]]

Technical / Hardware requirements

2024-05-15T07:09:18Z

Gregory: /* Hardware requirements */

===Hardware requirements===

*Entry level physical or virtual server *dedicated to NEOSYS only* (in case of virtual machine, the host machine MUST be dedicated to NEOSYS and access given to support team, otherwise issues on the host machine will be untraceable by support team and NEOSYS will not be responsible for overall performance)
*1 x Quad core CPU (minimum)
*8 GB RAM (minimum)
*Minimum 100GB SSD Enterprise Write intensive (recommend 2x for mirroring)

===Software requirements===

*Ubuntu Server 22.04 LTS
*Configured with an internal static IP (ie dynamic IP delivered by DHCP is not allowed)
*Strictly no additional software installed

===Web Browser on User Workstations===

Firefox, Chrome or MS Edge.

Browsers must have pop-up blockers turned off and must not have any 3rd party toolbars installed, that can potentially interfere with the NEOSYS user interface.

===Backup requirements===
If entry level physical server or if virtual server that supports USB passthrough (e.g. VMWare vSphere version 4.1 and above):

*3 x 16 GB USB drives for backup
*Someone to change the USB drives on a scheduled basis (weekly)

If virtual server that does not support USB passthrough:

*Client must *at their own responsibility* agree to arrange copy of the backup files to an external backup location every day.

Also refer to [[Backup_and_Restore#Backup_in_virtual_server| Backup to virtual server]]

===Router requirements===

*Port forwarding:
**SSH: port 19581 (router) -> port 19581 (server) for Host.
**SSH: port 19582 (router) -> port 19582 (server) for lxc container.
**HTTPS: port 4431 (router) -> port 4431 (server) (Required only if NEOSYS needs to be accessed from outside office and prior management approval for this is obtained)
*Access to send email via an internal email server

===Remote support requirement===

[[Letter_to_obtain_agreement_of_client_IT_staff_to_provide_remote_access| Letter to obtain agreement of client IT staff to provide remote access]]

Technical / Hardware requirements

2023-10-12T11:25:05Z

Gregory: /* Router requirements */

===Hardware requirements===

*Entry level physical or virtual server *dedicated to NEOSYS only* (in case of virtual machine, the host machine MUST be dedicated to NEOSYS and access given to support team, otherwise issues on the host machine will be untraceable by support team and NEOSYS will not be responsible for overall performance)
*1 x Quad core CPU (minimum)
*8 GB RAM (minimum)
*Minimum 100 GB HDD Enterprise Write intensive SSD (2 SSDs recommended, for mirroring)

===Software requirements===

*Ubuntu Server 22.04 LTS
*Configured with an internal static IP (ie dynamic IP delivered by DHCP is not allowed)
*Strictly no additional software installed

===Web Browser on User Workstations===

Firefox, Chrome or MS Edge.

Browsers must have pop-up blockers turned off and must not have any 3rd party toolbars installed, that can potentially interfere with the NEOSYS user interface.

===Backup requirements===
If entry level physical server or if virtual server that supports USB passthrough (e.g. VMWare vSphere version 4.1 and above):

*3 x 16 GB USB drives for backup
*Someone to change the USB drives on a scheduled basis (weekly)

If virtual server that does not support USB passthrough:

*Client must *at their own responsibility* agree to arrange copy of the backup files to an external backup location every day.

Also refer to [[Backup_and_Restore#Backup_in_virtual_server| Backup to virtual server]]

===Router requirements===

*Port forwarding:
**SSH: port 19581 (router) -> port 19581 (server) for Host.
**SSH: port 19582 (router) -> port 19582 (server) for lxc container.
**HTTPS: port 4431 (router) -> port 4431 (server) (Required only if NEOSYS needs to be accessed from outside office and prior management approval for this is obtained)
*Access to send email via an internal email server

===Remote support requirement===

[[Letter_to_obtain_agreement_of_client_IT_staff_to_provide_remote_access| Letter to obtain agreement of client IT staff to provide remote access]]

Configuring Thunderbird

2022-12-09T12:06:02Z

Gregory: /* Getting access to NEOSYS support calendar */

==Setting Up the Inbox Accounts==

The instructions given below explain how to set up the "Support" account on Thunderbird.

1. Open Mozilla Thunderbird. Choose the "Create a new account" option. Click on "Skip this and use my existing email" option on the "Welcome to Thunderbird" window.

[[File:Thunderbird_1.jpg]]

2. In the Mail Account Setup window, enter "support@neosys.com" for Your Name, "support@neosys.com" for Email Address and the password provided by Neosys staff. Click on continue.

[[File:Thunderbird_2.jpg]]

3. Click on "Manual Config".

[[File:Thunderbird_3.jpg]]

4. Key in "imap.neosys.com" for incoming IMAP host name, server hostname "smtp.neosys.com" and port "587" for outgoing SMTP and "support.neosys" as username. For SMTP, change the SSL to "STARTTLS" and authentication to "No authentication". Click on the "Re-test" button and then click "Done".

[[File:Thunderbird_4.jpg]]

BEFORE clicking on inbox and BEFORE adding other inboxes, you MUST refer to [[Configuring_Thunderbird#Adding_email_accounts_with_large_numbers_of_emails | Adding email accounts with large numbers of emails]]. This is because if you add all the accounts together at once, they will all sync together and it will take 1-2 days for the sync to complete.

For backups@neosys.com, support2@neosys.com (Nagios), name.neosys@gmail.com repeat the steps mentioned above, with the below-mentioned replacements.

1. For backups@neosys.com, enter "backups@neosys.com" for Your Name, "backups@neosys.com" for Email address and enter the password provided by Neosys staff.
Enter the username as "backups.neosys".

2. For support2@neosys.com, enter "support2@neosys.com" for Your Name, "support2@neosys.com" for Email address and enter the password provided by Neosys staff.
Enter the username as "support2.neosys".

3. For name.neosys@gmail.com, enter your full name for Your Name, "name.neosys@gmail.com" for Email address and enter the password. You do not need to follow the steps for manual config as the settings will be automatically configured.

[[File:thunderbird_6.jpg]]

[[File:thunderbird_7.jpg]]

==Account Settings==

Verify that the account settings for all the accounts are entered as shown below.

[[File:acsettings1.jpg]]

[[File:acsettings2.jpg]]

[[File:acsettings3.jpg]]

[[File:acsettings4.jpg]]

[[File:Account junk.jpg|500px]]

For backups@neosys.com account, under "Synchronization and Storage", change the setting to "Delete messages more than 365 days old"

The Synchronisation & Storage settings should be changed to the settings shown below only at a convenient time, since downloading all the emails will take a lot of time for accounts with large number of emails. Also see [[Configuring_Thunderbird#Adding_email_accounts_with_large_numbers_of_emails |Adding email accounts with large number of emails]]

[[File:acsettings6.jpg]]

[[File:acsettings7.jpg]]

[[File:acsettings8.jpg]]

===Adding email accounts with large numbers of emails===

After adding email accounts with large numbers of emails, in order to avoid long wait while downloading all the emails, it is best to change the account settings - BEFORE clicking on INBOX, which triggers the download/sync - to download the bodies of only the last 30 days of emails. The email headings are always synced but that does not take long.

See Account Settings, Synchronisation and Storage.

Since searching of emails only works when the bodies of emails have been downloaded, if you need to text search all emails, then, at a convenient time, you can remove or increase the number of days in the setting and somehow trigger the full sync.

[[File:thla.jpg]]

===Configuring Outgoing SMTP Mail Servers===

By default Thunderbird adds a new smtp profile with each Inbox added.

However you only need two, main and backup.

1. Right click on any Inbox > Settings.

2. Scroll down and click "Outgoing Server (SMTP)".

3. Delete all smtp server profiles except the default and ensure your settings match below screenshot.
[[File:Smtpsettings.png]]

4. Add a new profile with settings in below screenshot.

TODO - setup backup imap2 container and add settings screenshot here. (old imap2 smtp port 2500)

==Add-ons==

#Install addons: General settings menu (Top right 3 bars) > Addons
#Search and install the addons in below screenshot.
#Configure Mailbox Alert by Right e.g support inbox > Mailbox Alert:
##Check default sound and message.
[[File:Addon2.png]]

==Other General Configurations==

===Configure inbox to display selected columns===

The below options MUST be set in Support Inbox. To do so Right click on a column heading and select the below options:

#Thread
#Starred
#Attachment
#Subject
#Correspondents
#Received: You MUST add column RECEIVED which is the email date received to email inbox because the default Date seems to be the DATE SENT whereas we are primarily interested in the date we received the email. The difference is due to delays in email servers or spam tricks.

The below options MUST NOT be set in Support Inbox

#Junk status: Emails will automatically be moved to Junk folder if the junk icon in this column is clicked. Hence remove Junk Status column so that emails are not accidentally moved to junk. To remove, right-click on a column heading and untick "Junk Status".

===Configure Mail Content===

Preferences > Privacy > untick "Allow remote content in messages"

Support MUST set the "Allow remote content in messages" as blocked in Thunderbird without adding any exceptions for any email id. The reason is remote content i.e embedded images, stylesheets etc is a privacy concern as it sends your private information to the mail sender. We do not know the source of these embedded content in emails so cannot trust them as they can be web bugs.

===Configure Layout View===

Layout view should be set as Vertical view.

===Thunderbird Preferences===

#Preferences > Display > Advanced > untick " Automatically mark messages as read".
#Keep settings as below for Preferences > Security.
#Keep settings as below for Preferences > Composition > Spelling.

[[File:Globaljunk.jpg|500px]]

[[File:Composition.png|500px]]

===Configure MailAlert===

Right click Inbox > Mailbox Alert > Edit Mailbox Alert alerts > Select Default message. Set as below. Once set Right click Inbox > Mailbox Alert > check Default message

[[file:thsh.jpg]]

===Configure Auto Resize Image===

Set the preferences for the Auto Resize Image add-on as shown in the screenshot at the end of this section.

To handle different scenarios when resizing images in emails, follow the below steps:

*Resize all images in the email: Click "Send" followed by "Confirm" on the Auto resize screen.

*Skip resize all images in the email: Click "Send" followed by "Cancel Resizing" on the Auto resize screen.

*Resize Images only in the email trail and skip resizing new images: Click "Resize" on top of your screen and proceed with writing the email/adding new images. When ready to send the email, follow above step for skipping resize all images in the email.

[[File:Resize2.png]]

===Configure Header Tools Lite===

Set the preferences for the Header Tools Lite add-on as shown in the screenshot below.

[[File:headertoolslite.jpg]]

===Configure Dorando Keyconfig===

Menu > Add-ons > Edit Keyconfig Preferences > Disable shortcuts for 'Archive' and 'Junk' as shown in the screenshots below.

[[File:Keyconfig1.png]]

[[File:Keyconfig2.png]]

===Getting access to NEOSYS support calendar===

After confirming that the steps below are successfully done, delete your local personal calendar and make the shared support calendar the default.

In Thunderbird, add a new CalDAV network calendar by navigating to File (Alt+f) -> New -> Calendar.

[[File:Cal1.png]]

Location: https://nextcloud.hosts.neosys.com:44318/remote.php/dav/calendars/support/support_shared_by_steve/

You will have to enter your nextcloud user and pass.

[[File:Cal2.png]]

[[File:Cal3.png]]

It should show a tick if successful, otherwise you will have to Unsubscribe and try again.

[[File:Cal4.png]]

Change the new calendar properties to Refresh every 5 mins.

[[File:Cal5.png]]

===Applying filter for backup failure emails===

In order to filter any backup failure emails as Important, go to Tools > Message Filters and create a new filter with settings as shown below:

Refer to filters list of other staff members.

===Tags===

Find the prefs.js file in your profile directory:
find ~/.thunderbird/ -name prefs.js
Edit the prefs.js file
nano /home/gregory/.thunderbird/s3bqjf24.dsfaefawe/prefs.js
Replace the all the "user_pref("mailnews.tags." with the new set of tag configurations below and restart Thunderbird.
<pre>
user_pref("mailnews.tags.$label1.color", "#FF0000");
user_pref("mailnews.tags.$label1.tag", "Important");
user_pref("mailnews.tags.$label2.color", "#FF9900");
user_pref("mailnews.tags.$label2.tag", "Work");
user_pref("mailnews.tags.$label3.color", "#009900");
user_pref("mailnews.tags.$label3.tag", "Joel");
user_pref("mailnews.tags.$label4.color", "#3333ff");
user_pref("mailnews.tags.$label4.tag", "Greg");
user_pref("mailnews.tags.$label5.color", "#993399");
user_pref("mailnews.tags.$label5.tag", "Arvind");
user_pref("mailnews.tags.$label6.color", "#ff6e6e");
user_pref("mailnews.tags.$label6.tag", "Follow up");
user_pref("mailnews.tags.$label7.color", "#eb007b");
user_pref("mailnews.tags.$label7.tag", "Client issue");
user_pref("mailnews.tags.$label8.color", "#28e2d8");
user_pref("mailnews.tags.$label8.tag", "Delay support");
</pre>

==[[Procedures#Email_Retention_Policy| Email Retention Policy]]==

==[[Procedures#Handling_junk.2Fspam_email| Handling junk/spam emails]]==

==Viewing long running processes on Thunderbird==
When setting up and configuring Thunderbird, using its Activity Manager (Tools > Activity Manager) and possibly using the header bar "Always on top" option on the Activity Manager window may be helpful to know what long running processes are being performed.

[[File:Activity manager.jpg]]

==Deleting Thunderbird accounts==
Check the option "Remove Message Data" when removing accounts from Thunderbird. Otherwise, the old messages stay forever taking space on the disk.

[[File:dt2.png]]

Select Remove Message Data option as shown below.

[[File:dt.png]]

==Clearing out old improperly deleted Thunderbird Accounts==

If you delete Thunderbird accounts without checking the option "Remove Message Data" the old messages stay forever taking space on the disk.

To locate any such unnecessary files

ls -l -h /home/*/.thunderbird/*.default/ImapMail/*/INBOX

<pre>

-rw------- 1 arvind arvind 16M May 25 16:36 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.gmail.com/INBOX
-rw------- 1 arvind arvind 668M Mar 29 08:54 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-1.com/INBOX
-rw------- 1 arvind arvind 242M May 28 14:45 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-2.com/INBOX
-rw------- 1 arvind arvind 755M May 25 12:39 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-3.com/INBOX
-rw------- 1 arvind arvind 420M Apr 24 10:27 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-4.com/INBOX
-rw------- 1 arvind arvind 13G May 28 14:37 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys.com/INBOX

</pre>

Check the dates, which indicate the date of the last message received, to get a rough idea of what accounts are in use and not in use

Check which of the folders are currently in use by looking in Thunderbird settings for each account one by one, "Server Settings", "Local Directory"

Delete the folders and files which are not in use using whatever method you like.

".thunderbird", like all files and folders which start with a dot, is a hidden folder which you can make visible in Nautilus File Manager by pressing Ctrl+h

==Troubleshooting high CPU usage when idling==

If Thunderbird consumes 100+% CPU when there is no activity seen on Thunderbird activity manager (Tools -> Activity Manager), then try the below steps from https://kb.mozillazine.org/Performance_-_Thunderbird :

#Go to Thunderbird preferences -> General
#Scroll to the bottom and click "Config Editor..."
#Search for mail.db.idle_limit
#Change its value to 30000000 (7 zeroes, not 5 zeroes) and save
#Restart Thunderbird

EXODUS Knowledge

2022-11-21T06:51:54Z

Gregory: /* Debugging var values */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
General work for server services/configs. i.e /etc/nagios, /etc/apache2, /var/log/mail.log or general networking.

1: /root/exodus/
Exodus core is installed. Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus library source code which emulate the AREV system e.g var.cpp

3: /root/exodus/cli/src
Exodus Command Line Interface E.g edir, edic, list, listfiles, delete, createdb, deletedb.

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
The www/3/exodus has generic .htm, php and .js which provide basic website functionality i.e Login page, Support Menu, User details, Authorization.
NEOSYS www files linked into ./www, from /root/neosys/web, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
The dat/ has generic dictionary files. (See Section Dictionaries & Psql Functions)
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation & management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
/root/neosys has the neosys .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)

==Dictionaries & Psql Functions==

Dictionaries - data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) Field for raw data E.g "Office 44 Building Rush"
(S) Symbolic or calculated E.g for total ans=field1+field2
(G) General or Group E.g "@CRT" which contains always contains the default columns to show on the output of the list program; if no columns provided in list command.
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Delete Database dictionary file===

cli syncdat deletes database files if the dat dir is empty or contains a file 'SYNCDAT_DELETE'

<pre>
touch neosys/src/dat/dict.users/SYNC_DELETE
neosys/src~: syncdat
</pre>

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file (i.e dict.file record)=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

=====Delete dat record =====

Simply open the corresponding dat file, remove all lines and save.

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 4 variables of type std::string (var_str), int (var_int), long double (var_dbl) and another int (var_typ i.e flags).

The first 3 bits of var_typ (flags_) are used to signify which of the other variables are assigned.

First bit = string is un/assigned. Second bit = int is un/assigned. Third bit = double un/assigned (a combination of these bits describes whether var_str/ var_int/ var_dbl are assigned.

Examples:

000 (binary) = 0 (decimal) = non are assigned.

100 (binary) = 1 (decimal) = var_str is assigned. (var_int & var_dbl are not)

010 (binary) = 2 (decimal) = var_int is assigned. (var_str & var_dbl are not)

001 (binary) = 4 (decimal) = var_dbl is assigned. (var_str var_int are not)

Combinations:

110 (binary) = 3 (decimal) = var_str & var_int are assigned. (var_dbl is not)

====To inspect a “common” variable====

e,g, a variable called ba.ledgern

p ((subExodusProgram::ba_common)ba).ledgern

====To inspect a common block====

e.g. a common block ba

ptype (subExodusProgram::ba_common)ba

====To dump all variables in a common====

e.g all variables in ba

p (subExodusProgram::ba_common)ba

====What types are available in a program?====

<pre>info types common
info types subExodus
info types libexodus</pre>

====Other commands====

explore subExodusProgram::ba_common

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

READ: remember there is no test version of libexodus programs. This means that if libexodus programs are compiled and installed, neosys/src and exodus/services LIVE programs MUST BE UPGRADED.

#Upgrade test system to test
##If a dev system like de2, git stash any uncommitted changes other staff may be working on.
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live" in ~/neosys (to upgrade exodus/service and neosys/src live programs)
##Begin testing
#Upgrade clients
##*Client hosted - Schedule a planned upgrade with client.
##*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
##Day of upgrade, check USB backup was successful & make a container snapshot, otherwise do not proceed.
##Stop services
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live".
##Check services started automatically without issues.
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-11-21T06:35:35Z

Gregory: Improvement to var description

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
General work for server services/configs. i.e /etc/nagios, /etc/apache2, /var/log/mail.log or general networking.

1: /root/exodus/
Exodus core is installed. Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus library source code which emulate the AREV system e.g var.cpp

3: /root/exodus/cli/src
Exodus Command Line Interface E.g edir, edic, list, listfiles, delete, createdb, deletedb.

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
The www/3/exodus has generic .htm, php and .js which provide basic website functionality i.e Login page, Support Menu, User details, Authorization.
NEOSYS www files linked into ./www, from /root/neosys/web, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
The dat/ has generic dictionary files. (See Section Dictionaries & Psql Functions)
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation & management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
/root/neosys has the neosys .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)

==Dictionaries & Psql Functions==

Dictionaries - data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) Field for raw data E.g "Office 44 Building Rush"
(S) Symbolic or calculated E.g for total ans=field1+field2
(G) General or Group E.g "@CRT" which contains always contains the default columns to show on the output of the list program; if no columns provided in list command.
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Delete Database dictionary file===

cli syncdat deletes database files if the dat dir is empty or contains a file 'SYNCDAT_DELETE'

<pre>
touch neosys/src/dat/dict.users/SYNC_DELETE
neosys/src~: syncdat
</pre>

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file (i.e dict.file record)=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

=====Delete dat record =====

Simply open the corresponding dat file, remove all lines and save.

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 4 variables of type std::string (var_str), int (var_int), long double (var_dbl) and another int (var_typ i.e flags).

The first 3 bits of var_typ (flags_) are used to signify which of the other variables are assigned.

First bit = string is un/assigned. Second bit = int is un/assigned. Third bit = double un/assigned (a combination of these bits describes whether var_str/ var_int/ var_dbl are assigned.

Examples:

000 (binary) = 0 (decimal) = non are assigned.

110 (binary) = 3 (decimal) = var_str & var_int are assigned.

001 (binary) = 4 (decimal) = var_dbl is assigned.

====To inspect a “common” variable====

e,g, a variable called ba.ledgern

p ((subExodusProgram::ba_common)ba).ledgern

====To inspect a common block====

e.g. a common block ba

ptype (subExodusProgram::ba_common)ba

====To dump all variables in a common====

e.g all variables in ba

p (subExodusProgram::ba_common)ba

====What types are available in a program?====

<pre>info types common
info types subExodus
info types libexodus</pre>

====Other commands====

explore subExodusProgram::ba_common

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

READ: remember there is no test version of libexodus programs. This means that if libexodus programs are compiled and installed, neosys/src and exodus/services LIVE programs MUST BE UPGRADED.

#Upgrade test system to test
##If a dev system like de2, git stash any uncommitted changes other staff may be working on.
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live" in ~/neosys (to upgrade exodus/service and neosys/src live programs)
##Begin testing
#Upgrade clients
##*Client hosted - Schedule a planned upgrade with client.
##*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
##Day of upgrade, check USB backup was successful & make a container snapshot, otherwise do not proceed.
##Stop services
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live".
##Check services started automatically without issues.
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-10-27T05:26:21Z

Gregory: /* Development and deployment using 'dat' files */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
General work for server services/configs. i.e /etc/nagios, /etc/apache2, /var/log/mail.log or general networking.

1: /root/exodus/
Exodus core is installed. Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus library source code which emulate the AREV system e.g var.cpp

3: /root/exodus/cli/src
Exodus Command Line Interface E.g edir, edic, list, listfiles, delete, createdb, deletedb.

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
The www/3/exodus has generic .htm, php and .js which provide basic website functionality i.e Login page, Support Menu, User details, Authorization.
NEOSYS www files linked into ./www, from /root/neosys/web, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
The dat/ has generic dictionary files. (See Section Dictionaries & Psql Functions)
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation & management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
/root/neosys has the neosys .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)

==Dictionaries & Psql Functions==

Dictionaries - data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) Field for raw data E.g "Office 44 Building Rush"
(S) Symbolic or calculated E.g for total ans=field1+field2
(G) General or Group E.g "@CRT" which contains always contains the default columns to show on the output of the list program; if no columns provided in list command.
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Delete Database dictionary file===

cli syncdat deletes database files if the dat dir is empty or contains a file 'SYNCDAT_DELETE'

<pre>
touch neosys/src/dat/dict.users/SYNC_DELETE
neosys/src~: syncdat
</pre>

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file (i.e dict.file record)=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

=====Delete dat record =====

Simply open the corresponding dat file, remove all lines and save.

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

====To inspect a “common” variable====

e,g, a variable called ba.ledgern

p ((subExodusProgram::ba_common)ba).ledgern

====To inspect a common block====

e.g. a common block ba

ptype (subExodusProgram::ba_common)ba

====To dump all variables in a common====

e.g all variables in ba

p (subExodusProgram::ba_common)ba

====What types are available in a program?====

<pre>info types common
info types subExodus
info types libexodus</pre>

====Other commands====

explore subExodusProgram::ba_common

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

READ: remember there is no test version of libexodus programs. This means that if libexodus programs are compiled and installed, neosys/src and exodus/services LIVE programs MUST BE UPGRADED.

#Upgrade test system to test
##If a dev system like de2, git stash any uncommitted changes other staff may be working on.
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live" in ~/neosys (to upgrade exodus/service and neosys/src live programs)
##Begin testing
#Upgrade clients
##*Client hosted - Schedule a planned upgrade with client.
##*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
##Day of upgrade, check USB backup was successful & make a container snapshot, otherwise do not proceed.
##Stop services
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live".
##Check services started automatically without issues.
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-10-13T05:55:23Z

Gregory: /* Copy all programs and 'dat' files to ~/live/bin|lib|dat */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
General work for server services/configs. i.e /etc/nagios, /etc/apache2, /var/log/mail.log or general networking.

1: /root/exodus/
Exodus core is installed. Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus library source code which emulate the AREV system e.g var.cpp

3: /root/exodus/cli/src
Exodus Command Line Interface E.g edir, edic, list, listfiles, delete, createdb, deletedb.

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
The www/3/exodus has generic .htm, php and .js which provide basic website functionality i.e Login page, Support Menu, User details, Authorization.
NEOSYS www files linked into ./www, from /root/neosys/web, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
The dat/ has generic dictionary files. (See Section Dictionaries & Psql Functions)
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation & management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
/root/neosys has the neosys .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)

==Dictionaries & Psql Functions==

Dictionaries - data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) Field for raw data E.g "Office 44 Building Rush"
(S) Symbolic or calculated E.g for total ans=field1+field2
(G) General or Group E.g "@CRT" which contains always contains the default columns to show on the output of the list program; if no columns provided in list command.
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

=====Delete dat record =====

Simply open the corresponding dat file, remove all lines and save.

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

====To inspect a “common” variable====

e,g, a variable called ba.ledgern

p ((subExodusProgram::ba_common)ba).ledgern

====To inspect a common block====

e.g. a common block ba

ptype (subExodusProgram::ba_common)ba

====To dump all variables in a common====

e.g all variables in ba

p (subExodusProgram::ba_common)ba

====What types are available in a program?====

<pre>info types common
info types subExodus
info types libexodus</pre>

====Other commands====

explore subExodusProgram::ba_common

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

READ: remember there is no test version of libexodus programs. This means that if libexodus programs are compiled and installed, neosys/src and exodus/services LIVE programs MUST BE UPGRADED.

#Upgrade test system to test
##If a dev system like de2, git stash any uncommitted changes other staff may be working on.
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live" in ~/neosys (to upgrade exodus/service and neosys/src live programs)
##Begin testing
#Upgrade clients
##*Client hosted - Schedule a planned upgrade with client.
##*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
##Day of upgrade, check USB backup was successful & make a container snapshot, otherwise do not proceed.
##Stop services
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live".
##Check services started automatically without issues.
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-10-13T05:16:03Z

Gregory: /* Copy all programs and 'dat' files to ~/live/bin|lib|dat */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
General work for server services/configs. i.e /etc/nagios, /etc/apache2, /var/log/mail.log or general networking.

1: /root/exodus/
Exodus core is installed. Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus library source code which emulate the AREV system e.g var.cpp

3: /root/exodus/cli/src
Exodus Command Line Interface E.g edir, edic, list, listfiles, delete, createdb, deletedb.

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
The www/3/exodus has generic .htm, php and .js which provide basic website functionality i.e Login page, Support Menu, User details, Authorization.
NEOSYS www files linked into ./www, from /root/neosys/web, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
The dat/ has generic dictionary files. (See Section Dictionaries & Psql Functions)
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation & management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
/root/neosys has the neosys .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)

==Dictionaries & Psql Functions==

Dictionaries - data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) Field for raw data E.g "Office 44 Building Rush"
(S) Symbolic or calculated E.g for total ans=field1+field2
(G) General or Group E.g "@CRT" which contains always contains the default columns to show on the output of the list program; if no columns provided in list command.
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

=====Delete dat record =====

Simply open the corresponding dat file, remove all lines and save.

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

====To inspect a “common” variable====

e,g, a variable called ba.ledgern

p ((subExodusProgram::ba_common)ba).ledgern

====To inspect a common block====

e.g. a common block ba

ptype (subExodusProgram::ba_common)ba

====To dump all variables in a common====

e.g all variables in ba

p (subExodusProgram::ba_common)ba

====What types are available in a program?====

<pre>info types common
info types subExodus
info types libexodus</pre>

====Other commands====

explore subExodusProgram::ba_common

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

READ: remember there is no test version of libexodus programs. This means that if libexodus programs are compiled and installed, neosys/src and exodus/services LIVE programs MUST BE UPGRADED.

#Upgrade test system to test
##If a dev system like de2, git stash any uncommitted changes other staff may be working on.
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live" in ~/neosys (to upgrade exodus/service and neosys/src live programs)
##Begin testing
#Upgrade clients
##*Client hosted - Schedule a planned upgrade with client.
##*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
##Day of upgrade, check USB backup was successful & make a container snapshot, otherwise do not proceed.
##Stop services
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live".
##Check services started automatically without issues.
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-10-07T07:47:00Z

Gregory: /* Screens */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
General work for server services/configs. i.e /etc/nagios, /etc/apache2, /var/log/mail.log or general networking.

1: /root/exodus/
Exodus core is installed. Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus library source code which emulate the AREV system e.g var.cpp

3: /root/exodus/cli/src
Exodus Command Line Interface E.g edir, edic, list, listfiles, delete, createdb, deletedb.

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
The www/3/exodus has generic .htm, php and .js which provide basic website functionality i.e Login page, Support Menu, User details, Authorization.
NEOSYS www files linked into ./www, from /root/neosys/web, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
The dat/ has generic dictionary files. (See Section Dictionaries & Psql Functions)
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation & management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
/root/neosys has the neosys .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)

==Dictionaries & Psql Functions==

Dictionaries - data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) Field for raw data E.g "Office 44 Building Rush"
(S) Symbolic or calculated E.g for total ans=field1+field2
(G) General or Group E.g "@CRT" which contains always contains the default columns to show on the output of the list program; if no columns provided in list command.
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

====To inspect a “common” variable====

e,g, a variable called ba.ledgern

p ((subExodusProgram::ba_common)ba).ledgern

====To inspect a common block====

e.g. a common block ba

ptype (subExodusProgram::ba_common)ba

====To dump all variables in a common====

e.g all variables in ba

p (subExodusProgram::ba_common)ba

====What types are available in a program?====

<pre>info types common
info types subExodus
info types libexodus</pre>

====Other commands====

explore subExodusProgram::ba_common

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

READ: remember there is no test version of libexodus programs. This means that if libexodus programs are compiled and installed, neosys/src and exodus/services LIVE programs MUST BE UPGRADED.

#Upgrade test system to test
##If a dev system like de2, git stash any uncommitted changes other staff may be working on.
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live" in ~/neosys (to upgrade exodus/service and neosys/src live programs)
##Begin testing
#Upgrade clients
##*Client hosted - Schedule a planned upgrade with client.
##*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
##Day of upgrade, check USB backup was successful & make a container snapshot, otherwise do not proceed.
##Stop services
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live".
##Check services started automatically without issues.
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-10-07T07:36:08Z

Gregory: /* Dictionaries & Psql Functions */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)

==Dictionaries & Psql Functions==

Dictionaries - data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) Field for raw data E.g "Office 44 Building Rush"
(S) Symbolic or calculated E.g for total ans=field1+field2
(G) General or Group E.g "@CRT" which contains always contains the default columns to show on the output of the list program; if no columns provided in list command.
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

====To inspect a “common” variable====

e,g, a variable called ba.ledgern

p ((subExodusProgram::ba_common)ba).ledgern

====To inspect a common block====

e.g. a common block ba

ptype (subExodusProgram::ba_common)ba

====To dump all variables in a common====

e.g all variables in ba

p (subExodusProgram::ba_common)ba

====What types are available in a program?====

<pre>info types common
info types subExodus
info types libexodus</pre>

====Other commands====

explore subExodusProgram::ba_common

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

READ: remember there is no test version of libexodus programs. This means that if libexodus programs are compiled and installed, neosys/src and exodus/services LIVE programs MUST BE UPGRADED.

#Upgrade test system to test
##If a dev system like de2, git stash any uncommitted changes other staff may be working on.
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live" in ~/neosys (to upgrade exodus/service and neosys/src live programs)
##Begin testing
#Upgrade clients
##*Client hosted - Schedule a planned upgrade with client.
##*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
##Day of upgrade, check USB backup was successful & make a container snapshot, otherwise do not proceed.
##Stop services
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live".
##Check services started automatically without issues.
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-06-02T07:48:39Z

Gregory: /* Upgrading to new version */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries & Psql Functions==

Dictionaries, data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) field for raw data E.g "Office 44 Building Rush"
(S) symbolic or calculated E.g for total ans=field1+field2
(G) General, pending
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

READ: remember there is no test version of libexodus programs. This means that if libexodus programs are compiled and installed, neosys/src and exodus/services LIVE programs MUST BE UPGRADED.

#Upgrade test system to test
##If a dev system like de2, git stash any uncommitted changes other staff may be working on.
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live" in ~/neosys (to upgrade exodus/service and neosys/src live programs)
##Begin testing
#Upgrade clients
##*Client hosted - Schedule a planned upgrade with client.
##*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
##Day of upgrade, check USB backup was successful & make a container snapshot, otherwise do not proceed.
##Stop services
##Run "g" in ~/exodus (to do git pull --ff-only)
##Run "m" in ~/exodus (to compile and install libexodus)
##Run "~/neosys/upgrade.all.sh live".
##Check services started automatically without issues.
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-06-02T07:06:43Z

Gregory: /* ==./copyall lists programs you have not patched */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries & Psql Functions==

Dictionaries, data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) field for raw data E.g "Office 44 Building Rush"
(S) symbolic or calculated E.g for total ans=field1+field2
(G) General, pending
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched=====
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

Use when there are too many changes to do individually patches.

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-06-02T07:06:27Z

Gregory: /* =Troubleshoot */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries & Psql Functions==

Dictionaries, data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) field for raw data E.g "Office 44 Building Rush"
(S) symbolic or calculated E.g for total ans=field1+field2
(G) General, pending
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot====

=====./copyall lists programs you have not patched===
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

Use when there are too many changes to do individually patches.

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-06-02T07:06:02Z

Gregory: /* Single Patches */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries & Psql Functions==

Dictionaries, data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) field for raw data E.g "Office 44 Building Rush"
(S) symbolic or calculated E.g for total ans=field1+field2
(G) General, pending
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Patching neosys/src or exodus/service/src===

#Manually insert patches and compile into test programs.
#If patch dat files then run neosys/compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic
*#login to BASIC_TEST and test
*Other clients:
*#./test DBCODE
*#login to DBCODE and test patches works
#run exodus/service/copyall and check only the patched programs are listed. (if not, see Troubleshoot below)
#Inform users to log off for 10mins. As a rule of thumb, live services must be stopped before copying test programs to live.
#exodus/service/copyall CONFIRM

====Troubleshoot===
=====./copyall lists programs you have not patched===
Use exodus/service/copyone to copy only your patched programs one by one. (./copyone will not copy dat files)

===Upgrading to new version===

Use when there are too many changes to do individually patches.

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

Configuring NEOSYS Generally

2022-05-20T08:20:42Z

Gregory:

==System Configuration File==

See [http://userwiki.neosys.com/index.php/System_Configuration_File System Configuration File]

==Clearing files in database==

This is to be done if you want to clean an old database or clean a training database so that a client can enter fresh data.

These commands DO NOT reset the data to "factory settings" so for new installations you need to download a fresh BACKUP.ZIP file from the NEOSYS website.

*Clear transactions F5 - CLEAROP (only clears transactions not reference files)

*Clear all F5 - CLEARALL (rather nasty command because it clears all reference files as well)

*Clear finance transactions F5 - CLEARACC

REINDEXALL SHOULD be done after CLEAROP or CLEARACC in order to reduce the size of the index files to reflect the reduction in size of data, by rebuilding them from scratch, since index files don't reduce in size automatically when the data is cleared or reduced.

==Clearing selected files in database==

This can be done if you want to reimport selected files.

WARNING This procedure must not be used if there are any transaction in the system that might use the files being cleared.

WARNING Clearing individual files is error prone because other files might refer to the records you are clearing (referential integrity is not applied so dangling references may be created). For example, clearing suppliers without clearing vehicles in advance results in vehicles which have supplier codes that do not exist. Even if you reimport the suppliers, some supplier codes in the vehicles file may not be reimported leaving vehicles with problems.

WARNING The opportunity to create problems that have no solution and that may only surface when the system is in operation is endless unless you think through the implications very very very carefully.

*Flush Index F5 FLUSH.INDEX (this command MUST be performed immediately after any data clearing, because CLEARFILE does not clear the related indexes immediately and it will be done later on access, causing random stopping later on)

*Clear suppliers F5 CLEARFILE SUPPLIERS

*Clear vehicles F5 CLEARFILE VEHICLES

If a particular field needs to be cleared from a file then get the exact field name defined in the file.

To list the various fields in a file F5 LD CLIENTS (for Client and Brand file)

*Clear Payment Instruction from Client and Brand file F5 CLEARFIELD CLIENTS PAYMENT_INSTRUCTIONS (only clears the payment instruction defined in all the Client & Brand file)

*Clear Agency fee % from Client & Brand file F5 CLEARFIELD CLIENTS FEE (only clears the agency fee defined for all Client & Brand flies)

==Mass updating database without data entry==

Warning: It is advisable that you take the approval of NEOSYS DBA or programmers before doing any of the following procedures. There is no protection whatsoever from damaging the database if you do not appreciate all the implications of any particular update. Common sense and caution must be used. If you damage a database then it may be, or with operation become, irretrievably damaged and require reconstruction from a backup causing possibly extreme finance damages to the owner of the data and consequences for yourself. You have been warned.

There are many commands in maintenance mode that allow you to amend the database directly and without any record and without any ability to reverse changes.

Normally, no record of the changes is made. All changes will appear to have been done by the last user at the time and date of the last normal user interface amendments.

===Available fields to clear or set===

#Client & Brand File: CLIENTS MARKET_CODE
#Client & Brand File: BRANDS MARKET_CODE

===How to clear a database field===

Assuming that a particular database field may be blank (i.e. not required for data entry) then you may clear a field as follows.

Warning: There is nothing to stop you clearing a field that is mandatory and doing this may cause irrecoverable damage to the database.

In the following example we wish to change all clients with market code “UAE to have market code blank.

First, if you don’t want to clear all records, “select” the required records.

SELECT CLIENTS WITH MARKET_CODE “UAE”

After a period of time, depending on the number of records in the file, it should briefly state the number of records selected and then return to the command prompt.

WARNING: If no records have been selected then ALL records will be updated by the following command!

CLEARFIELD CLIENTS MARKET_CODE

===How to set a database field===

In the following example we change all the clients where the market code is blank (has not been entered) to become “UAE”.

Warning: You can set the market code to a market code that does not exist. This will cause various problems in the operation of the system but is probably not irrecoverable.

First, if you don’t want to set all records, “select” the required records.

SELECT CLIENTS WITH MARKET_CODE “”

After a period of time, depending on the number of records in the file, it should briefly state the number of records selected and then return to the command prompt.

WARNING: If no records have been selected then ALL records will be updated by the following command!

CLEARFIELD CLIENTS MARKET_CODE/UAE

==Backup to other media (i.e. not to USB)==

[[Backup and Restore#Backup to other media (i.e. not to USB)|Backup to other media]]

==Reordering databases in maintenance mode==

#Open NEOSYS in maintenance mode
#General > Backup & data management > Reorder databases
#Press the Enter key to select/deselect the databases in the required order.
#Press F9 once and confirm that the databases are in the required order, if not go back to Step 3.
#Press F9 again and select Yes to save the reordered list.

==Copying a single record from one database to another==

You need to know the file name and record key of the record to be copied.

In this case the file is DEFINITIONS and the key is AGENCY.PARAMS

You can invent any old style 8.3 filename instead of C:\AGP.DAT in the following example

On the source computer:

F5
COPY DEFINITIONS AGENCY.PARAMS TO: (DOS C:\AGP.DAT)

On the target computer:

F5
COPY DOS C:\AGP.DAT (ON) TO: (DEFINITIONS AGENCY.PARAMS)

The (O) option is required to force overwrite of the existing

The (N) option means only copy if the target already exists. It is advisable to use it when you know that the target already exists to avoid misspellings in the command. It must be omitted if the target doesnt exist.

==Allowing users temporary login as NEOSYS in maintenance mode==

#Get them to login with any name even NEOSYS
#Get them to enter "?" for the pass without the quotes
#NEOSYS will give them a lock like "NEOSYS 123456" which they must give you. You should not log out until the next step is completed
#Follow the NEOSYS lock/key procedure using the full contents of the lock including the user name

(to allow access EXCEPT access to authorisation screen use a special number (not documented here) as the last number of the initial command)

#Give them the key and get them to enter and proceed

==Configuring upload of photoshop "cs2" jpg files==

Photoshop version "cs2" produces jpg files that cannot be viewed in Internet Explorer.

A solution is to rename the files extension from .jpg to .psjpg before uploading.

"psjpg" files are an invention of NEOSYS and IIS must be configured to handle .psjpg files as follows:

Windows Server 2003 (doesnt work on XP)

#Computer Management, Internet Information Server, Properties
#Click MIME Types
#Click New
#Extension: psjpg
#MIME Type: application/photoshop
#Click OK,OK,OK
#Restart IIS (Right click, All Tasks, Restart)

==Configuring next Media Booking Order No.==

It is not possible to edit the next booking order number, as done with other document sequences, within the Invoice Number screen.

The CURRENT booking order number is defined in the below definitions record.

EXO_DATA=<dbcode> edir definitions BOOKING_ORDERS.SK

EXODUS Knowledge

2022-04-11T06:35:44Z

Gregory: /* Psql unaccent extension */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries & Psql Functions==

Dictionaries, data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) field for raw data E.g "Office 44 Building Rush"
(S) symbolic or calculated E.g for total ans=field1+field2
(G) General, pending
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

===Psql unaccent extension===

The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

====Error====
<pre>
MVDBException:ERROR: function immutable_unaccent(text) does not exist
ERROR: permission denied to create extension "unaccent"
HINT: Must be superuser to create this extension.
</pre>

====Solution====
sudo -u postgres psql -c 'ALTER USER exodus WITH SUPERUSER;'

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Single Patches===

#Insert patches into various files and compile if necessary
#If patch dat files then run neosys compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic #requires /etc/systemd/system/agy_test@.service to have been patched like agy_live@.service
*#web login to BASIC_TEST to test
*If not patsalides
*#./test DBCODE
*#login to DBCODE and test
#If all ok
#./copyall
#CHECK what WILL be copied!
#Judge if really need to get people off (unlikely for typical patch) and request users to logout for 10 mins
#./copyall CONFIRM and CHECK what was copied is as per exceptions same as previous step
#./copyone could be used instead of copyall but it has no test mode and does not copy dat files

===Upgrading to new version===

Use when there are too many changes to do individually patches.

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-04-11T06:01:06Z

Gregory: /* Dictionaries & Psql Functions */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries & Psql Functions==

Dictionaries, data used to describe other data; specifically the fields of other files.

There are three types of fields: <pre>
(F) field for raw data E.g "Office 44 Building Rush"
(S) symbolic or calculated E.g for total ans=field1+field2
(G) General, pending
</pre>

Show dictionary using "EXO_DATA=test list dict.clients".

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

===Psql unaccent extension===
The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Single Patches===

#Insert patches into various files and compile if necessary
#If patch dat files then run neosys compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic #requires /etc/systemd/system/agy_test@.service to have been patched like agy_live@.service
*#web login to BASIC_TEST to test
*If not patsalides
*#./test DBCODE
*#login to DBCODE and test
#If all ok
#./copyall
#CHECK what WILL be copied!
#Judge if really need to get people off (unlikely for typical patch) and request users to logout for 10 mins
#./copyall CONFIRM and CHECK what was copied is as per exceptions same as previous step
#./copyone could be used instead of copyall but it has no test mode and does not copy dat files

===Upgrading to new version===

Use when there are too many changes to do individually patches.

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-04-11T05:59:03Z

Gregory:

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries & Psql Functions==

Dictionaries, data used to describe other data; specifically the fields of other files.

There are three types of fields:
(F) field for raw data E.g "Office 44 Building Rush"
(S) symbolic or calculated E.g for total ans=field1+field2
(G) General, pending

Dictionary files in Exodus are now stored in dat files.

===Development and deployment using 'dat' files===

====Rationale====

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

===='dat' files====

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

====Location of dat files====

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

====Editing and deploying a 'dat' file====

It is currently a three step process to edit and deploy such 'dat' files.

=====Edit the 'dat' file=====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

=====Copy all 'dat' files to ~/dat =====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

=====Copy all programs and 'dat' files to ~/live/bin|lib|dat=====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

===Psql unaccent extension===
The unaccent extension and the immutable_unaccent function are created in the db ‘template1’ during exodus installation while user is postgres which is a superuser.

Thereafter all new databases are created as copies of template1 so they contain the extension and function

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Upgrading Exodus==

#Develop patch in Dev system

===Single Patches===

#Insert patches into various files and compile if necessary
#If patch dat files then run neosys compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic #requires /etc/systemd/system/agy_test@.service to have been patched like agy_live@.service
*#web login to BASIC_TEST to test
*If not patsalides
*#./test DBCODE
*#login to DBCODE and test
#If all ok
#./copyall
#CHECK what WILL be copied!
#Judge if really need to get people off (unlikely for typical patch) and request users to logout for 10 mins
#./copyall CONFIRM and CHECK what was copied is as per exceptions same as previous step
#./copyone could be used instead of copyall but it has no test mode and does not copy dat files

===Upgrading to new version===

Use when there are too many changes to do individually patches.

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

Troubleshooting NEOSYS Finance System

2022-03-23T09:10:16Z

Gregory: /* Fixing "Cross Check Balance" warnings using CHK.ALLOC */

==Using Finance Maintenance Mode fixing tools==
<div style="color:red">

Follow the instructions below before you proceed with fixing CCB errors with any check/fix:

Do not do mass fixes eg “Add/Delete ALL xyz” options from a year in the past eg 2007 while only checking accounts for 2009. You must CHECK EVERYTHING THAT YOU REQUEST TO BE FIXED.

The various fixing tools like CHK.ALLOC etc are impossible to make safe in all circumstances so if you don’t really know what you are doing you assume the WORST not the BEST with these fixing tools.

''CHECK EVERYTHING YOU “FIX”!!!''

Do not go back risking fixes in prior years for which Finance has been closed, unless absolutely necessary.

While checking/fixing a CCB error, check and fix any other errors if present to avoid CCB's in future.

After doing check/fixes, all accounts should be tested for cross check balance errors. Do this by taking ledger printout for the whole year for all accounts.

IF YOU DO NOT FOLLOW INSTRUCTIONS CAREFULLY YOU CAN END UP MODIFYING AUDITED ACCOUNTS AND IT WILL TAKE DAYS FOR PROGRAMMERS TO TRY AND CORRECT THE DAMAGE YOU CAUSE OR EVEN REQUIRE A RESTORE OF A BACKUP CAUSING LOSS OF MANY DAYS WORK TO THE CLIENT
</div>

==Cross-Year Cross Check Balance Warnings==

When you use Opening Balance Journals, NEOSYS does not guarantee that the opening balances of one year match the closing balance of a prior year. This occurs quite normally and commonly occurs in the situations described below.

Regardless of the cause, in such cases, any statement or ledger account of movements (not open items) that crosses the disjoint years will have a "cross check balance" note at the bottom of the account. A cross check balance warning is just a warning that the closing balance of the account (as calculated from a simple total of the opening balance in the old year plus all transactions) does not agree with the actual account balance according to the trial balance in the new year.

If the opening balance of the account in the second year is not equal to the closing balance in the prior year then logically the two years cannot be viewed as a single continuous account and agree with the trial balance.

===Posting into years prior to Opening Balance===

When you start up a company in one year using Opening Balance Journals then the closing balances of the prior years remain zero. If you thereafter start to post into prior years, the closing balances of those years will remain in disagreement with the opening balance of the following years.

'''Solution'''

If you wish to obtain cross-year accounts between years, and you wish such account's balances to agree with the later year trial balance, then you must arrange for the closing balances of the prior years to agree with the opening balances of the following years.

To do this, ensure that any Opening Balance Journals that are posted into any particular year are also posted into all prior financial years of interest, or better, post them directly into the earliest year and any following years will automatically be updated as normal.

===Opening Balance Journals===

If you post Opening Balance Journals into one year for any reason, they are not posted into prior years. Therefore the opening balances of the amended year become different from the prior year.

In this respect, auditors amendments to opening balances should be posted as normal journals in the final period of the prior year and not as Opening Balance Journals in the current year.

===Producing Historical Accounts===

As mentioned above, if you post Opening Balance Journals into one year, they are not posted into prior years. For such accounts, NEOSYS will show Cross Check Balance warning if you take a ledger account or statement of account report across multiple years including years prior to the Opening Balance Journal posting. e.g. if an account has opening balance entry in 2011, and no entries in 2010, then ledger account report with period setting 2010 - 2018 will show Cross Check Balance warning.

'''Solution'''

To avoid this Cross Check Balance warning, in the report period settings exclude years prior to the Opening Balance Journal posting, since the account anyway has no entries in the prior years.

=== Producing P&L and Retained Earnings Accounts ===
Refer to [https://userwiki.neosys.com/index.php/Troubleshooting_NEOSYS_Finance_System#Cross_Check_Balance_error_on_all_Profit_and_Loss_accounts_and_Retained_Earnings_account CCB on P&L and Retained Earnings accounts]

==Fixing "Cross Check Balance" warnings using CHK.ALLOC==

Cross Check Balance Message on an account means the total of the outstanding items in an account does not match the balance as in the movement - typically the outstanding shows a higher balance than the balance in the movement, but might not always be the case. The reasons why this may happen is because of the following:

*Reposting journals containing allocated items (which NEOSYS doesnt handle in all circumstances)
*Failing to clear open item accounts on a regular basis where the number of postings is high

This procedure only applies to CCB on the outstanding item type statements which are the more common problem. CCB on movement accounts are rarer, more serious and cannot be fixed by this procedure.

This might not always fix the CCB warnings and hence you will need to escalate to the programmer.

<pre>
!!! WARNING !!!

This is a dangerous procedure.
IF YOU DO NOT FOLLOW INSTRUCTIONS CAREFULLY YOU CAN END UP MODIFYING AUDITED ACCOUNTS AND AUDITORS/ACCOUNTANTS AND IT WILL TAKE DAYS FOR PROGRAMMERS TO TRY AND CORRECT THE DAMAGE YOU CAUSE!

DO THIS ON TESTDATA FIRST
</pre>

This can be run while other users are online.

While running the command, monitor the lines coming up on the screen for any warnings. In case you make any changes and select "Yes" or "Yes to all" for any message, you should re run the command again until clean.

This exact procedure restores vouchers missing from open item accounts:

*Exodus TEST Database
<pre>
1. /root/neosys/test <DBCODE>
2. After startup press "x" key.
3. Type: "chkalloc"
</pre>
*Exodus LIVE Database
<pre>
1. Stop all LIVE processes. (pressing x won't return prompt to enter check command if other LIVE processes running)
2. Run /root/neosys/run <DBCODE>
3. After startup press "x" key.
4. Type: "chkalloc"
</pre>
*AREV Both LIVE and TEST databases
<pre>
F5
UTIL (nothing happens at this stage as its a background process and gives you access to the next command)
CHK.ALLOC
</pre>

Minimum voucher year.period?

Generally go back as little as possible to cover vouchers that might be missing from O/I accounts.

Which account do you want or blank for all?

You MUST search for all accounts. Entering one account WILL NOT WORK since it will only check vouchers already on the open item accounts and we are looking for those that are missing.

IGNORE OR RESPOND NEGATIVELY to all messages or questions EXCEPT the following:

Missing from O/I index?

Select "Yes" or "Yes to all"

===Meaning of other messages===

1. The following message pair does not usually cause cross check balances and indicates that they have cancelled some foreign currency allocation since allocation should not cause and change in rate.

IN*175510*C 20/01/08 ALMA 4/2 REC*4440*C allocation base outstanding -7326.15 expected -6762.60 (-563.550,.08333333)
IN*175510*C 20/01/08 ALMA 4/2 REC*4440*C allocation changed exchange rate from .08333333 to .07692308

2. For the following message, you MUST choose "Leave all" , otherwise risk damage to closed audited prior year accounts:

Account numbers do not agree. Delete the allocation?

3. For the following message, choose "Add it to other voucher" ONLY for the voucher/account that needs fixing. Do not choose "Add all" unless you know each and every allocation that will be added, as it could cause problems such as allocating to vouchers that are already allocated i.e. causing double allocations. You MUST check everything that you request to be fixed.

Allocation is missing ?

4. For the following message, choose "Skip further Warnings":

Amounts do not agree - please fix manually

===Re-enabling CCB warning mail notifications===
NEOSYS is pre-configured to send out email notifications (to the same group of people who receive the backup alerts) when a Cross Check Balance (CCB) warning is found for the first time on an account. That means there will be only 1 email notification irrespective of the times the warning occurs until it is fixed.

After the CCB is fixed or after clearing or otherwise eliminating the same, you need to delete the CCB file in D:\neosys\neosys to re-enable NEOSYS to resend notification by email after it discovers CCB warnings on that account again in the future.

==Fixing "Cross Check Balance" warning using CHK.VINDEX==

YOU MUST READ AND APPLY THE GENERAL INSTRUCTIONS WRITTEN FOR CHK.ALLOC AS WELL FOR CHK.VINDEX

This program can fix a few CCB warnings that CHK.ALLOC cannot - including some in balance forward type accounts (ie not open item/outstanding item accounts) .

Not all the possible fixes are described here and any messages which are not described here should be responded to NEGATIVELY!

CCB error for [http://userwiki.neosys.com/index.php/Troubleshooting_NEOSYS_Finance_System#Fixing_Error:_.22A.2Fc._.3F.3F.3F.22_message_in_account_of_outstanding_items Wrong A/C] can be fixed by running CHK.VINDEX

===Different account - Delete the index entry?===
<pre>
"*MEVEJ*FZ" L=232 V=REC*461*FZ 05/07/09 B=447 "MEVEJ"
different account "ADVOPME"
Delete the index entry?
───┬─────────────────────────────────────────────────────────────
1>No
2│None
3│Yes
4│All
</pre>

Select Yes.

===Missing Voucher - Delete the index entry?===
<pre>
╔═════════════════════════════════════════════════════════════════╗
║ "*VIV*N" L=458 V=IN*2012/1170*N 29/05/12 missing voucher ║
║ Delete the index entry? ║
║───┬─────────────────────────────────────────────────────────────║
║ 1>No ║
║ 2│Yes ║
║ 3│None ║
║ 4│All ║
╚═════════════════════════════════════════════════════════════════╝
</pre>

Select Yes.

===Different Date - Correct index entry?===
<pre>
"*OMD*AP" L=115 V=IN*2013/350*AP 31/03/13 B=1159
different date "14/03/13"
Correct index entry?
───┬─────────────────────────────────────────────────────────────
1>No
2│Yes
3│Correct None
4│Correct All
</pre>

Select Yes.

==Checking if a voucher is fully posted==

Sometimes due to some kind of system error a batch is not fully posted then one voucher in the batch may be half-posted. Any remaining vouchers in the batch (after/below) will almost certainly be completely unposted.

To see if this has happened, first identify which voucher is problematic and then check if the first and last lines of the voucher are correctly posted by seeing if the voucher appears in their respective ledger accounts.

===First line is not posted===

If the first line does not appear then probably the whole voucher is not posted at all.

Reposting the voucher may successfully solve the problem but see [http://techwiki.neosys.com/index.php/Troubleshooting_NEOSYS_Finance_System#Reposting_a_Batch_in_Journal_Entry Reposting a batch] and [http://techwiki.neosys.com/index.php/Troubleshooting_NEOSYS_Finance_System#Fixing_.22Cross_Check_Balance.22_warnings_using_CHK.ALLOC Cross Check Balance] to avoid potential issues in reposting.

===Last line is not posted===

If the first line appears but the last line does not appear as posted in its ledger account then the voucher is "HALF POSTED". This breaks the all important law of double entry accounting and the trial balance will not balance.

Reposting the voucher will not solve the problem because the system does not take into account that the voucher is half posted when it is reversing the original voucher during the repost.

===First and last lines are posted===

In the case there is no problem. The voucher is fully posted and the half-posted issue is not present

==Voucher number missing from posted batches==

In some cases voucher numbers are missing from posted batches.

There could be several reasons why the voucher numbers are missing. The voucher number is generated but does not reflect on the batch or the voucher number did not get generated at the time of posting. We are still investigating the root cause to this problem.

For both cases, take a ledger print out for the account the voucher number is missing and check if the voucher number is present in the ledger print out.

===1. The entries in the batch reflect on the ledger print===

If the entries reflect in the ledger print out, open the voucher file and investigate why the voucher number did not appear when the batch was posted.

The voucher numbers will appear in the batch when the batch is re-saved. The same voucher numbers appears as it was in the voucher file.

===2. The entry in the batch does not reflect on the ledger print===

If the voucher file is not present in the ledger print out, open the batch and investigate why the voucher number did not generate when the batch was posted.

The voucher numbers will be generated when the batch is re-saved but not in sequence number. Notify clients the batches are re-saved and the voucher number will not be in sequence.

===Solution===

Run CHK.VOUCHERS and check for errors. Then re-save the batch.

The authorisation to re-save a batch is restricted to NEOSYS and a few client users at the moment.

====Running CHK.VOUCHERS====
Run the following command in maintenance mode and check for errors.

F5
UTIL
CHK.VOUCHERS

When running CHK.VOUCHERS, respond to prompts as shown below. Generally respond negatively to prompts which are not specified below, but in TEST datasets you can experiment by responding positively.

Choose "Yes" for the following message:

Missing from batch but matching ref with no voucher can be found. Add it?

Choose "No" for the following message:

Delete empty unposted / deleted voucher ?

Choose "Yes or All" for the following message:

Account “ XYZ” type is “ COST” , but the voucher analysis code “28*1*COMPANYCODE*XYZ” is type “ BILL/INCOME” .
Fix voucher analysis code to be COST and repost? :---- YES/ALL

Choose "All" for the following message. This is to keep the file "clean" of messages but does not actually cause any changes in the rest of the system.

Account "XYZ" is "JOBS", but the voucher analysis code "28*1*clientcode**marketcode*suppliercode*mediatypecode" is "MEDIA"
Fix voucher analysis code is to be JOBS and repost?

After doing check/fix, all accounts should be tested for cross check balance errors. Do this by taking ledger printout for the whole year for all accounts.

==[http://userwiki.neosys.com/index.php/Troubleshooting_NEOSYS_Finance_System#Why_does_an_outstanding_not_appear_while_taking_a_ledger_printout Fixing CCB errors that do not get fixed with CHK.ALLOC, CHK.VINDEX or CHK.VOUCHERS]==

==Using CHK.POST to fix some types of Cross Check Balance (CCB) errors==

CHK.POST can be used to fix/recreate the BALANCES files. This could be useful in the following cases:

#Some stubborn Cross Check Balance errors
#BALANCES file is damaged (use [[Handling_damaged_files#Using_FIXFILE_to_repair_corrupted_files|FIXFILE]] first)

Use it strictly only after ALL other errors have been fixed using the usual CHK programs

*CHK.VOUCHERS
*CHK.VINDEX
*CHK.ALLOC
*CHK.BATCHES
*CHK.BALANCES
*CHK.ACCOUNTS
*CHK.CHARTS
*CHK.CONTROLS
*CHK.OB (As of 12/2/19: Only Bates and AdlineD can parameters be entered in UI, whereas in TEST and other clients, editing of program code must be done to tailor the program to check what you want. E.g a period)

[[Troubleshooting_NEOSYS_Finance_System#Check_steps_of_each_CHK_program|What checks are performed on in each CHK routine]]

This program adjusts the balances in the trial balance reports (which also show in the “Cross Check Balance phrase on the Detailed Ledger Accounts) to match the vouchers found in the voucher file. Note that CHK.ALLOC and CHK.VINDEX have no effect on these figures and instead amend the transactions shown on the Detailed Ledger Account - and thereby the totals on the same report so that they agree with the CCB amount and thereby the conflict is resolved.

Most CCB in NEOSYS are related to problems on outstanding item accounts and are fixed with the CHK.ALLOC program. More rarely, problems occur in the movement accounts and are fixed with the CHK.VINDEX program - although this can also fix some errors in outstanding item statements. Even more rarely will the problem be on the Trial Balances/CCB balances and can be fixed with this CHK.POST procedure

WARNING! This solution may make things worse so don’t use on live data unless EXHAUSTIVELY checked that everything is ok on test data first.

WARNING! Don’t go back and “correct” previous years which have been closed because auditors expect them never to change (EVEN IF THEY ARE WRONG!) and it can be impossible to put them back “wrong” if you “correct” them.

WARNING! If you correct previous years then you must [[Troubleshooting_NEOSYS_Finance_System#Redo_Open_New_Year_procedure| rerun the Open New Year procedure]] for all prior years.

#In maintenance mode press Alt+1
#Enter the range of periods that you want to check/adjust the trial balance for e.g. 1/9-7/9 for 1st Period of 2009 up to the 7th period of 2009.
#Press F9 and Esc
#Press F5 and type CHK.POST
#What stage to start at? Choose “Select Vouchers”
#Clear the updated balances file? Choose “Clear it”
#OK to Start? … OK
#If there are any discrepancies found between the Trial Balance Balances/CCB balances … and the Vouchers in the file you will get some fairly cryptic questions asking you, one by one, if you want to fix the balances.

IMPORTANT

#It is advised that you only fix the balances which refer to the accounts that you are concerned about. You will find the account number buried in the cryptic questions referred to above.
#If you fix subsidiary account balances then MAKE SURE that you also fix the control accounts balances (if you are prompted).

==B10 & B12 Errors==

The error might be displayed as follows :

ERROR NO: B10 IN DAYBOOK.SUBSX AT 133
Variable has not been assigned a value. Zero used.

This error should have said something like : "The ledgers are closed up to the period you have just tried to post" OR "Financial Year 2011 must be opened before you post into it."

Try again and you would see the exact message if you have not opened the new year as yet.

==Fixing "Index has been deleted" message==

===Error Message===

The index record "TEXT.XREF*abc*xyz"
has been deleted.
The index for "TEXT.XREF"
should be rebuilt.

===Solution===

[http://techwiki.neosys.com/index.php/Handling_damaged_files#Fixing_damaged_index_files_.28names_starting_with_.21.29 Perform REINDEXVOUCHERS in maintenance mode].

==Reposting a Batch in Journal Entry==
The authorisation to repost a batch is restricted to NEOSYS and a few client users at the moment. Reposting a batch will repost all the vouchers in the batch.

See [http://userwiki.neosys.com/index.php/Troubleshooting_NEOSYS_Finance_System#Correcting_mistakes_in_postings Correcting mistakes in postings] for NEOSYS policy for correcting mistakes in journal postings.

Warning: Reposting journals which are already allocated will create cross check balance errors, so before reposting, any allocations should be de-allocated to avoid these errors.

Reposting a batch should be done in TEST dataset first. After reposting, all accounts should be tested for cross check balance errors. Do this by taking ledger printout for the whole year for all accounts.

See [[Procedures#Amending.2FReposting_Journal_Entries|Procedure for Reposting Journal Entries]]

To repost a batch, edit the journal on the Journal Entry/Query page and save. Click OK for the prompt “OK to repost now?”.
==Editing and Reposting Vouchers==

WARNING: Can cause effectively irreversible damage to the finance database requiring restoration of a suitable backup causing loss of data and/or total loss of database if backup not available.

It is almost impossible to edit and repost vouchers and keep the audit trail correct so this procedure is of limited use. The stages are UNPOST, EDIT, REPOST.

Must be done on test data and advisable to run CHK.VOUCHERS afterwards.

===UNPOST===
F5
UNPOST XXX*YYYYY*ZZZ

Where:

*XXX is the voucher type/journal code
*YYY is the voucher number
*ZZZ is the company code

Any problems with the voucher might generate messages but the problems will be reverse the problems which must have occurred when posting the voucher in the first place.

===EDIT===
F5
ED VOUCHERS XXX*YYYYY*ZZZ

For example, if due to a neosys system error, the ZZZ999 suspense account has been used instead of some missing internal system accounts like exchange gain/loss accounts then you can insert the correct account.

Change line 8 (which contains the account numbers) from:

REZO²REZO²BMDI²BMDI²²

to

REZO²REZO²BMDI²BMDI²EXDI²EXDI

where EXDI is the ORIGINAL A/c No. of the Exchange Difference A/c.

F9, ESC to save and exit

===REPOST===

F5
REPOST XXX*YYYYY*ZZZ

==Removing strange outstanding items on accounts that have balance of zero==

Sometimes running CHK.ALLOC on periods too far back in time causes old items to be incorrectly resurrected and become outstanding.

If the account balance is zero in the current period and has been for some years then you can simply delete all outstanding items.

Assuming the (original) account number is AAAA and the company code is CC, in maintenance mode, F5.

DELETE VOUCHER.INDEX *AAAA*CC

It should say "1 Item(s) deleted"

==Reinstating Open Item Statements after Clear Open Items==

WARNING: DO THIS PROCESS ON TEST DATA FIRST TO VERIFY THAT IT DOES NOT CAUSE CROSS CHECK BALANCE ERRORS

WARNING: THIS PROCESS MAY CAUSE CROSS CHECK BALANCE ERRORS BY REINSTATING "ANCIENT CORRUPT VOUCHERS"

"CHECK BACK" AS FEW YEARS AS NEEDED TO INCLUDE ALL VOUCHERS THAT COULD BE OPEN/OUTSTANDING AT THE DESIRED PERIOD.

It is not easy to know how far to "check back" for open items so go back two years initially unless it is known that there may be items outstanding from even earlier years.

===1. Check, solve, record all existing cross check balances===

Restoring open items may create cross check balance errors so first you need to fix all EXISTING cross check balance errors in the database.

Do not proceed to the next step until this step is complete otherwise you will have no idea what problems were created by CHK.ALLOC and what problems were in the database before.

Expect no sympathy from programmers if you ignore this or fail to work on testdata first.

First print ***ALL*** open item ledger accounts to check for existing cross check balances in open item accounts.

Correct all "cross check balances" using the usual procedures to do so.

Note that cross check balances are unavoidable on some *extremely large* open item accounts. This does not mean that you can ignore cross check balances on merely large accounts. They must be fixed.

Save copies of the detailed ledger accounts of any *extremely large" open items accounts that you have not fixed their cross check balances.

===2. Edit back the Company File "cleared-upto period"===

Complete this step quickly if users are working online - in order not to lock the company file for longer than it takes to change the "cleared upto period".

In maintenance mode, press F5. example company code X. Do this for all companies that need to.

ED COMPANIES X

change line 17 to be the last period (YYMM) desired to have been cleared.

DONT add or delete any lines!

F9, Esc

===3. Run CHK.ALLOC===

This process can be run while users are working since there is a only a very slight chance that they will be posting the same account at the time that the program is updating its open item voucher index.

In maintenance mode press F5

CHK.ALLOC

Minimum is how far to go back. See explanation above.
<pre>
╔═══════════════════════════════════════╗
║ Minimum voucher year.period to check? ║
║ (0 for all) ║
║<200801 >║
╚═══════════════════════════════════════╝
</pre>

Which account do you want? PRESS ESC (or Enter+Esc)
<pre>
╔═════════════════════════════════════════╗
║ Which account do you want ? ║
║ ║
║ Give the account number or name, ║
║ or press [Enter] if not known. ║
║ (separate multiple entries with spaces) ║
║< >║
╚═════════════════════════════════════════╝
</pre>
Add it? ... Choose "Yes to All"
<pre>
╔═══════════════════════════╗
║ Missing from O/I index ║
║ Add it? (1.1KB) ║
║───┬───────────────────────║
║ 1│Yes ║
║ 2│No ║
║ 3>Yes to all <-THIS ONE ║
║ 4│No to all ║
║ 5│Yes to all *HAEX*FZ ║
║ 6│No to all *HAEX*FZ ║
╚═══════════════════════════╝
</pre>

===4. Check for cross check balances===

Print ***ALL*** open item ledger accounts to check for any additional cross check balances in open item accounts.

For speed, omit old WIPxx and ACCxx ledgers since their accounts are probably all closed.

If you have any ADDITIONAL cross check balances compared to those recorded in step 1, then you can rerun step 3 and "check back" a little earlier. Try one year earlier at a time - to avoid the problem of reinstating "ancient corrupt vouchers".

==Speeding up CHK.VOUCHERS CHK.ALLOC==

When running the above programs takes time due to a large number of vouchers, then you can speed up the process by prebuilding a filter and using it on each invocation of the program. This filter is lost when you login again.

For example, if you want to check vouchers for account numbers XXXX, YYYY and ZZZZ in financial period 2004.01 and onwards

SELECT VOUCHERS WITH YEAR_PERIOD GE "14.01" AND WITH ACCOUNT "XXXX" "YYYY" "ZZZZ"

wait for it to say "xxxx record(s) selected" briefly and then show blank command line again, then type

SAVELIST XYZ

then each time just before you run CHK.VOUCHERS and CHK.ALLOC you can do the following to make them only look at the selected vouchers

GETLIST XYZ

==Check steps of each CHK program==

===Run CHK.ALLOC===

Check the VOUCHERS file

Checks that allocations occur in pairs and that both sides refer to the same account number and are in the same financial period.

Checks not allocated more than the amount/base amount on the voucher.

Checks open items (not allocated or not fully allocated) exist in the open item records of the voucher index file.

Checks that allocations do not cause a rate change since automatic revaluation is supposed to occur on allocation.

===Run CHK.CONTROLS===

Checks the BALANCES file.

Check that the control account balances agree with the total of the balances of their subsidiary accounts.

===Run CHK.VOUCHERS===

Checks a lot of things in the VOUCHERS File.

===Run CHK.BATCHES===

Check the BATCHES file (Journals File).

Checks journal type, company code and currency codes are valid.

For each voucher check that it is not missing and that its voucher period, posted/unposted state and Journal No. agree with that of the Journal.

Checks amounts and base amounts are all numeric.

Checks base currency entries have matching base amount entries ie if base is USD then amount is 100USD and base is 100 (101 or something else).

For posted batches, checks A/c Nos. are valid.

===Run CHK.VINDEX===

Checks the VOUCHER.INDEX file.

For each entry in the index ie line on a ledger account.

Checks for ledger entries that have problems eg wrong account or missing voucher.

Checks A/c No, currency code, journal type and company code are valid.

Checks open items indexes are only for open item accounts.

Checks Journal type OB does not exist on movement indexes.

Checks that reversed batches have matching reversal batch and vice versa.

Checks vouchers exist.

Checks that the voucher period, date, currency and A/c No. agrees with the index.

===Run CHK.OB===

Check the BALANCES file.

Checks that opening balances are correct versus closing balances for accounts that are neither closed nor closing accounts.

===Run CHK.BALANCES===

Checks the BALANCES file.

Checks A/c No., currency code, voucher type and company code are all valid.

Checks balances are all numeric.

Checks the decimal places of balances of base currency are not more than the standard number of base currency decimal places.

Checks the base currency record references to all other currencies is alphabetical.

Checks all non-base currency balance records are referenced on the base currency record.

===Run CHK.CHARTS===

Checks the CHARTS and LEDGERS files.

Checks A/c and Original A/c are valid ie exist in ACCOUNTS file.

Checks any company and currency codes are valid.

Checks the internal lookup tables in DEFINITIONS file are correct.

===Run CHK.ACCOUNTS===

Checks the ACCOUNTS file.

Checks accounts exist in the Chart File.

Checks the account records representing Account and Orig A/c No. refer to each other symmetrically.

Checks that any Control A/c, Closing A/c, Company code and Currency code is valid and match the same in the Chart File.

==Fixing opening balance of new year does not agree with the closing balance of the previous year==

In order to fix this problem, redo the open new year procedure for the problematic year, where recreating opening balances (which is part of the opennewyear.subs program) should fix the issue.

===Redo Open New Year procedure===

In latest versions of NEOSYS, you can reopen a year using the below command.

REOPENYEAR <Companycodes> YYYY
<pre>
Media Production Finance Management Timesheets Files General Exit
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒╒═══════════════════════════════════TCL - 2══════════════════════════════════╕▒
▒│ │▒
▒│ :REOPENYEAR A,AA 2020 │▒
▒│ │▒
▒╘════════════════════════════════════════════════════════════════════════════╛▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
</pre>

In Maintenance mode, run CHK.OB and include a few prior years as well, to see if all balances are fine.
<pre>
Media Production Finance Management Timesheets Files General Exit
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒╒═══════════════════════════════════TCL - 2══════════════════════════════════╕▒
▒│ │▒
▒│ :chk.ob │▒
▒│ │▒
▒╘════════════════════════════════════════════════════════════════════════════╛▒
▒▒▒▒▒▒▒▒▒▒▒▒╔══════════════════════════════════════════════════════╗▒▒▒▒▒▒▒▒▒▒▒▒
▒▒▒▒▒▒▒▒▒▒▒▒║ Years to check? ║▒▒▒▒▒▒▒▒▒▒▒▒
▒▒▒▒▒▒▒▒▒▒▒▒║ (0 or starting year for all, but only checks forward ║▒▒▒▒▒▒▒▒▒▒▒▒
▒▒▒▒▒▒▒▒▒▒▒▒║ ie ob v cb not backward cb v ob) ║▒▒▒▒▒▒▒▒▒▒▒▒
▒▒▒▒▒▒▒▒▒▒▒▒║<15 16 17 18 19 >║▒▒▒▒▒▒▒▒▒▒▒▒
▒▒▒▒▒▒▒▒▒▒▒▒╚══════════════════════════════════════════════════════╝▒▒▒▒▒▒▒▒▒▒▒▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
</pre>

EXODUS Knowledge

2022-03-22T11:33:54Z

Gregory: /* Speedup postgresql importing databases */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries==
Dictionaries, the files used to describe the fields of a file's record.
Unlike in AREV, there is a copy of all dictionaries in each pgsql database (In AREV, updating a dictionary would affect all the databases).

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Updating a pgsql function in an exodus dictionary==

PENDING

==Development and deployment using 'dat' files==

===Rationale===

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

==='dat' files===

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

===Location of dat files===

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Editing and deploying a 'dat' file===

It is currently a three step process to edit and deploy such 'dat' files.

====Edit the 'dat' file====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

====Copy all 'dat' files to ~/dat ====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

====Copy all programs and 'dat' files to ~/live/bin|lib|dat====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

==Upgrading Exodus==

#Develop patch in Dev system

===Single Patches===

#Insert patches into various files and compile if necessary
#If patch dat files then run neosys compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic #requires /etc/systemd/system/agy_test@.service to have been patched like agy_live@.service
*#web login to BASIC_TEST to test
*If not patsalides
*#./test DBCODE
*#login to DBCODE and test
#If all ok
#./copyall
#CHECK what WILL be copied!
#Judge if really need to get people off (unlikely for typical patch) and request users to logout for 10 mins
#./copyall CONFIRM and CHECK what was copied is as per exceptions same as previous step
#./copyone could be used instead of copyall but it has no test mode and does not copy dat files

===Upgrading to new version===

Use when there are too many changes to do individually patches.

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

==Database Attach==
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-03-18T08:36:28Z

Gregory:

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries==
Dictionaries, the files used to describe the fields of a file's record.
Unlike in AREV, there is a copy of all dictionaries in each pgsql database (In AREV, updating a dictionary would affect all the databases).

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Updating a pgsql function in an exodus dictionary==

PENDING

==Development and deployment using 'dat' files==

===Rationale===

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

==='dat' files===

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

===Location of dat files===

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Editing and deploying a 'dat' file===

It is currently a three step process to edit and deploy such 'dat' files.

====Edit the 'dat' file====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

====Copy all 'dat' files to ~/dat ====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

====Copy all programs and 'dat' files to ~/live/bin|lib|dat====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

==Upgrading Exodus==

#Develop patch in Dev system

===Single Patches===

#Insert patches into various files and compile if necessary
#If patch dat files then run neosys compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./test basic #requires /etc/systemd/system/agy_test@.service to have been patched like agy_live@.service
*#web login to BASIC_TEST to test
*If not patsalides
*#./test DBCODE
*#login to DBCODE and test
#If all ok
#./copyall
#CHECK what WILL be copied!
#Judge if really need to get people off (unlikely for typical patch) and request users to logout for 10 mins
#./copyall CONFIRM and CHECK what was copied is as per exceptions same as previous step
#./copyone could be used instead of copyall but it has no test mode and does not copy dat files

===Upgrading to new version===

Use when there are too many changes to do individually patches.

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

===Database Attach===
<pre>
root@exodus:~/exodus/cli/src# dbattach
dbattach - Attach foreign database files to the current default database

Syntax is dbattach TARGETDB [FILENAME,...] {OPTION...}

Options:

F Forcibly delete any normal files in the current default database.

R Detach foreign files
RR Remove foreign database connection.
RRR Remove all foreign database connections and attachments.

L List attached foreign files in the current default database

If no filenames are provided then a database connection is created.
This is *required* before filenames can be provided and attached.

Notes:

Attachments are permanent until re-attached or removed.

Indexes on attached files behave strangely.
</pre>
Issues:

TARGETDB must be on the same connection as the current default database.

Current user must have access to the foreign database.

If EXO_USER and EXO_PASS are not set then exodus defaults are used.

exodus .config file is not used currently.

EXODUS Knowledge

2022-03-16T14:11:40Z

Gregory: /* Upgrading to new version */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries==
Dictionaries, the files used to describe the fields of a file's record.
Unlike in AREV, there is a copy of all dictionaries in each pgsql database (In AREV, updating a dictionary would affect all the databases).

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Updating a pgsql function in an exodus dictionary==

PENDING

==Development and deployment using 'dat' files==

===Rationale===

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

==='dat' files===

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

===Location of dat files===

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Editing and deploying a 'dat' file===

It is currently a three step process to edit and deploy such 'dat' files.

====Edit the 'dat' file====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

====Copy all 'dat' files to ~/dat ====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

====Copy all programs and 'dat' files to ~/live/bin|lib|dat====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

==Upgrading Exodus==

#Develop patch in Dev system

===Single Patches===

#Insert patches into various files and compile if necessary
#If patch dat files then run neosys compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./run basic
*#login to BASIC to test
*If not patsalides
*#./test DBCODE
*#login to DBCODE and test
#If all ok
#./copyall
#CHECK what WILL be copied!
#Judge if really need to get people off (unlikely for typical patch) and request users to logout for 10 mins
#./copyall CONFIRM and CHECK what was copied is as per exceptions same as previous step
#./copyone could be used instead of copyall but it has no test mode and does not copy dat files

===Upgrading to new version===

Use when there are too many changes to do individually patches.

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

EXODUS Knowledge

2022-03-16T14:09:48Z

Gregory: /* Upgrading Exodus */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries==
Dictionaries, the files used to describe the fields of a file's record.
Unlike in AREV, there is a copy of all dictionaries in each pgsql database (In AREV, updating a dictionary would affect all the databases).

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Updating a pgsql function in an exodus dictionary==

PENDING

==Development and deployment using 'dat' files==

===Rationale===

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

==='dat' files===

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

===Location of dat files===

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Editing and deploying a 'dat' file===

It is currently a three step process to edit and deploy such 'dat' files.

====Edit the 'dat' file====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

====Copy all 'dat' files to ~/dat ====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

====Copy all programs and 'dat' files to ~/live/bin|lib|dat====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

==Upgrading Exodus==

#Develop patch in Dev system

===Single Patches===

#Insert patches into various files and compile if necessary
#If patch dat files then run neosys compall to get dat files into ~/dat
*If Patsalides:
*#SITE_DIR=ptcy ./run basic
*#login to BASIC to test
*If not patsalides
*#./test DBCODE
*#login to DBCODE and test
#If all ok
#./copyall
#CHECK what WILL be copied!
#Judge if really need to get people off (unlikely for typical patch) and request users to logout for 10 mins
#./copyall CONFIRM and CHECK what was copied is as per exceptions same as previous step
#./copyone could be used instead of copyall but it has no test mode and does not copy dat files

===Upgrading to new version===

Many patches.

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

Configuring Thunderbird

2022-03-14T07:27:54Z

Gregory: /* Tags */

==Setting Up the Inbox Accounts==

The instructions given below explain how to set up the "Support" account on Thunderbird.

1. Open Mozilla Thunderbird. Choose the "Create a new account" option. Click on "Skip this and use my existing email" option on the "Welcome to Thunderbird" window.

[[File:Thunderbird_1.jpg]]

2. In the Mail Account Setup window, enter "support@neosys.com" for Your Name, "support@neosys.com" for Email Address and the password provided by Neosys staff. Click on continue.

[[File:Thunderbird_2.jpg]]

3. Click on "Manual Config".

[[File:Thunderbird_3.jpg]]

4. Key in "imap.neosys.com" for incoming IMAP host name, server hostname "smtp.neosys.com" and port "587" for outgoing SMTP and "support.neosys" as username. For SMTP, change the SSL to "STARTTLS" and authentication to "No authentication". Click on the "Re-test" button and then click "Done".

[[File:Thunderbird_4.jpg]]

BEFORE clicking on inbox and BEFORE adding other inboxes, you MUST refer to [[Configuring_Thunderbird#Adding_email_accounts_with_large_numbers_of_emails | Adding email accounts with large numbers of emails]]. This is because if you add all the accounts together at once, they will all sync together and it will take 1-2 days for the sync to complete.

For backups@neosys.com, support2@neosys.com (Nagios), name.neosys@gmail.com repeat the steps mentioned above, with the below-mentioned replacements.

1. For backups@neosys.com, enter "backups@neosys.com" for Your Name, "backups@neosys.com" for Email address and enter the password provided by Neosys staff.
Enter the username as "backups.neosys".

2. For support2@neosys.com, enter "support2@neosys.com" for Your Name, "support2@neosys.com" for Email address and enter the password provided by Neosys staff.
Enter the username as "support2.neosys".

3. For name.neosys@gmail.com, enter your full name for Your Name, "name.neosys@gmail.com" for Email address and enter the password. You do not need to follow the steps for manual config as the settings will be automatically configured.

[[File:thunderbird_6.jpg]]

[[File:thunderbird_7.jpg]]

==Account Settings==

Verify that the account settings for all the accounts are entered as shown below.

[[File:acsettings1.jpg]]

[[File:acsettings2.jpg]]

[[File:acsettings3.jpg]]

[[File:acsettings4.jpg]]

[[File:Account junk.jpg|500px]]

For backups@neosys.com account, under "Synchronization and Storage", change the setting to "Delete messages more than 365 days old"

The Synchronisation & Storage settings should be changed to the settings shown below only at a convenient time, since downloading all the emails will take a lot of time for accounts with large number of emails. Also see [[Configuring_Thunderbird#Adding_email_accounts_with_large_numbers_of_emails |Adding email accounts with large number of emails]]

[[File:acsettings6.jpg]]

[[File:acsettings7.jpg]]

[[File:acsettings8.jpg]]

===Adding email accounts with large numbers of emails===

After adding email accounts with large numbers of emails, in order to avoid long wait while downloading all the emails, it is best to change the account settings - BEFORE clicking on INBOX, which triggers the download/sync - to download the bodies of only the last 30 days of emails. The email headings are always synced but that does not take long.

See Account Settings, Synchronisation and Storage.

Since searching of emails only works when the bodies of emails have been downloaded, if you need to text search all emails, then, at a convenient time, you can remove or increase the number of days in the setting and somehow trigger the full sync.

[[File:thla.jpg]]

===Configuring Outgoing SMTP Mail Servers===

By default Thunderbird adds a new smtp profile with each Inbox added.

However you only need two, main and backup.

1. Right click on any Inbox > Settings.

2. Scroll down and click "Outgoing Server (SMTP)".

3. Delete all smtp server profiles except the default and ensure your settings match below screenshot.
[[File:Smtpsettings.png]]

4. Add a new profile with settings in below screenshot.

TODO - setup backup imap2 container and add settings screenshot here. (old imap2 smtp port 2500)

==Add-ons==

#Install addons: General settings menu (Top right 3 bars) > Addons
#Search and install the addons in below screenshot.
#Configure Mailbox Alert by Right e.g support inbox > Mailbox Alert:
##Check default sound and message.
[[File:Addon2.png]]

==Other General Configurations==

===Configure inbox to display selected columns===

The below options MUST be set in Support Inbox. To do so Right click on a column heading and select the below options:

#Thread
#Starred
#Attachment
#Subject
#Correspondents
#Received: You MUST add column RECEIVED which is the email date received to email inbox because the default Date seems to be the DATE SENT whereas we are primarily interested in the date we received the email. The difference is due to delays in email servers or spam tricks.

The below options MUST NOT be set in Support Inbox

#Junk status: Emails will automatically be moved to Junk folder if the junk icon in this column is clicked. Hence remove Junk Status column so that emails are not accidentally moved to junk. To remove, right-click on a column heading and untick "Junk Status".

===Configure Mail Content===

Preferences > Privacy > untick "Allow remote content in messages"

Support MUST set the "Allow remote content in messages" as blocked in Thunderbird without adding any exceptions for any email id. The reason is remote content i.e embedded images, stylesheets etc is a privacy concern as it sends your private information to the mail sender. We do not know the source of these embedded content in emails so cannot trust them as they can be web bugs.

===Configure Layout View===

Layout view should be set as Vertical view.

===Thunderbird Preferences===

#Preferences > Display > Advanced > untick " Automatically mark messages as read".
#Keep settings as below for Preferences > Security.
#Keep settings as below for Preferences > Composition > Spelling.

[[File:Globaljunk.jpg|500px]]

[[File:Composition.png|500px]]

===Configure MailAlert===

Right click Inbox > Mailbox Alert > Edit Mailbox Alert alerts > Select Default message. Set as below. Once set Right click Inbox > Mailbox Alert > check Default message

[[file:thsh.jpg]]

===Configure Auto Resize Image===

Set the preferences for the Auto Resize Image add-on as shown in the screenshot at the end of this section.

To handle different scenarios when resizing images in emails, follow the below steps:

*Resize all images in the email: Click "Send" followed by "Confirm" on the Auto resize screen.

*Skip resize all images in the email: Click "Send" followed by "Cancel Resizing" on the Auto resize screen.

*Resize Images only in the email trail and skip resizing new images: Click "Resize" on top of your screen and proceed with writing the email/adding new images. When ready to send the email, follow above step for skipping resize all images in the email.

[[File:Resize2.png]]

===Configure Header Tools Lite===

Set the preferences for the Header Tools Lite add-on as shown in the screenshot below.

[[File:headertoolslite.jpg]]

===Configure Dorando Keyconfig===

Menu > Add-ons > Edit Keyconfig Preferences > Disable shortcuts for 'Archive' and 'Junk' as shown in the screenshots below.

[[File:Keyconfig1.png]]

[[File:Keyconfig2.png]]

===Getting access to NEOSYS support calendar===

After confirming that the steps below are successfully done, delete your local personal calendar and make the shared support calendar the default.

In Thunderbird, add a new CalDAV network calendar by navigating to File (Alt+f) -> New -> Calendar.

You will have to enter your nextcloud user and pass.

[[File:Cal1.png]]

Location: https://nextcloud.hosts.neosys.com:44318/remote.php/dav/calendars/support/support_shared_by_steve/

[[File:Cal2.png]]

[[File:Cal3.png]]

It should show a tick if successful, otherwise you will have to Unsubscribe and try again.

[[File:Cal4.png]]

Change the new calendar properties to Refresh every 5 mins.

[[File:Cal5.png]]

===Applying filter for backup failure emails===

In order to filter any backup failure emails as Important, go to Tools > Message Filters and create a new filter with settings as shown below:

Refer to filters list of other staff members.

===Tags===

Find the prefs.js file in your profile directory:
find ~/.thunderbird/ -name prefs.js
Edit the prefs.js file
nano /home/gregory/.thunderbird/s3bqjf24.dsfaefawe/prefs.js
Replace the all the "user_pref("mailnews.tags." with the new set of tag configurations below and restart Thunderbird.
<pre>
user_pref("mailnews.tags.$label1.color", "#FF0000");
user_pref("mailnews.tags.$label1.tag", "Important");
user_pref("mailnews.tags.$label2.color", "#FF9900");
user_pref("mailnews.tags.$label2.tag", "Work");
user_pref("mailnews.tags.$label3.color", "#009900");
user_pref("mailnews.tags.$label3.tag", "Joel");
user_pref("mailnews.tags.$label4.color", "#3333ff");
user_pref("mailnews.tags.$label4.tag", "Greg");
user_pref("mailnews.tags.$label5.color", "#993399");
user_pref("mailnews.tags.$label5.tag", "Arvind");
user_pref("mailnews.tags.$label6.color", "#ff6e6e");
user_pref("mailnews.tags.$label6.tag", "Follow up");
user_pref("mailnews.tags.$label7.color", "#eb007b");
user_pref("mailnews.tags.$label7.tag", "Client issue");
user_pref("mailnews.tags.$label8.color", "#28e2d8");
user_pref("mailnews.tags.$label8.tag", "Delay support");
</pre>

==[[Procedures#Email_Retention_Policy| Email Retention Policy]]==

==[[Procedures#Handling_junk.2Fspam_email| Handling junk/spam emails]]==

==Viewing long running processes on Thunderbird==
When setting up and configuring Thunderbird, using its Activity Manager (Tools > Activity Manager) and possibly using the header bar "Always on top" option on the Activity Manager window may be helpful to know what long running processes are being performed.

[[File:Activity manager.jpg]]

==Deleting Thunderbird accounts==
Check the option "Remove Message Data" when removing accounts from Thunderbird. Otherwise, the old messages stay forever taking space on the disk.

[[File:dt2.png]]

Select Remove Message Data option as shown below.

[[File:dt.png]]

==Clearing out old improperly deleted Thunderbird Accounts==

If you delete Thunderbird accounts without checking the option "Remove Message Data" the old messages stay forever taking space on the disk.

To locate any such unnecessary files

ls -l -h /home/*/.thunderbird/*.default/ImapMail/*/INBOX

<pre>

-rw------- 1 arvind arvind 16M May 25 16:36 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.gmail.com/INBOX
-rw------- 1 arvind arvind 668M Mar 29 08:54 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-1.com/INBOX
-rw------- 1 arvind arvind 242M May 28 14:45 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-2.com/INBOX
-rw------- 1 arvind arvind 755M May 25 12:39 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-3.com/INBOX
-rw------- 1 arvind arvind 420M Apr 24 10:27 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-4.com/INBOX
-rw------- 1 arvind arvind 13G May 28 14:37 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys.com/INBOX

</pre>

Check the dates, which indicate the date of the last message received, to get a rough idea of what accounts are in use and not in use

Check which of the folders are currently in use by looking in Thunderbird settings for each account one by one, "Server Settings", "Local Directory"

Delete the folders and files which are not in use using whatever method you like.

".thunderbird", like all files and folders which start with a dot, is a hidden folder which you can make visible in Nautilus File Manager by pressing Ctrl+h

Configuring Thunderbird

2022-03-14T07:27:33Z

Gregory: /* Tags */

==Setting Up the Inbox Accounts==

The instructions given below explain how to set up the "Support" account on Thunderbird.

1. Open Mozilla Thunderbird. Choose the "Create a new account" option. Click on "Skip this and use my existing email" option on the "Welcome to Thunderbird" window.

[[File:Thunderbird_1.jpg]]

2. In the Mail Account Setup window, enter "support@neosys.com" for Your Name, "support@neosys.com" for Email Address and the password provided by Neosys staff. Click on continue.

[[File:Thunderbird_2.jpg]]

3. Click on "Manual Config".

[[File:Thunderbird_3.jpg]]

4. Key in "imap.neosys.com" for incoming IMAP host name, server hostname "smtp.neosys.com" and port "587" for outgoing SMTP and "support.neosys" as username. For SMTP, change the SSL to "STARTTLS" and authentication to "No authentication". Click on the "Re-test" button and then click "Done".

[[File:Thunderbird_4.jpg]]

BEFORE clicking on inbox and BEFORE adding other inboxes, you MUST refer to [[Configuring_Thunderbird#Adding_email_accounts_with_large_numbers_of_emails | Adding email accounts with large numbers of emails]]. This is because if you add all the accounts together at once, they will all sync together and it will take 1-2 days for the sync to complete.

For backups@neosys.com, support2@neosys.com (Nagios), name.neosys@gmail.com repeat the steps mentioned above, with the below-mentioned replacements.

1. For backups@neosys.com, enter "backups@neosys.com" for Your Name, "backups@neosys.com" for Email address and enter the password provided by Neosys staff.
Enter the username as "backups.neosys".

2. For support2@neosys.com, enter "support2@neosys.com" for Your Name, "support2@neosys.com" for Email address and enter the password provided by Neosys staff.
Enter the username as "support2.neosys".

3. For name.neosys@gmail.com, enter your full name for Your Name, "name.neosys@gmail.com" for Email address and enter the password. You do not need to follow the steps for manual config as the settings will be automatically configured.

[[File:thunderbird_6.jpg]]

[[File:thunderbird_7.jpg]]

==Account Settings==

Verify that the account settings for all the accounts are entered as shown below.

[[File:acsettings1.jpg]]

[[File:acsettings2.jpg]]

[[File:acsettings3.jpg]]

[[File:acsettings4.jpg]]

[[File:Account junk.jpg|500px]]

For backups@neosys.com account, under "Synchronization and Storage", change the setting to "Delete messages more than 365 days old"

The Synchronisation & Storage settings should be changed to the settings shown below only at a convenient time, since downloading all the emails will take a lot of time for accounts with large number of emails. Also see [[Configuring_Thunderbird#Adding_email_accounts_with_large_numbers_of_emails |Adding email accounts with large number of emails]]

[[File:acsettings6.jpg]]

[[File:acsettings7.jpg]]

[[File:acsettings8.jpg]]

===Adding email accounts with large numbers of emails===

After adding email accounts with large numbers of emails, in order to avoid long wait while downloading all the emails, it is best to change the account settings - BEFORE clicking on INBOX, which triggers the download/sync - to download the bodies of only the last 30 days of emails. The email headings are always synced but that does not take long.

See Account Settings, Synchronisation and Storage.

Since searching of emails only works when the bodies of emails have been downloaded, if you need to text search all emails, then, at a convenient time, you can remove or increase the number of days in the setting and somehow trigger the full sync.

[[File:thla.jpg]]

===Configuring Outgoing SMTP Mail Servers===

By default Thunderbird adds a new smtp profile with each Inbox added.

However you only need two, main and backup.

1. Right click on any Inbox > Settings.

2. Scroll down and click "Outgoing Server (SMTP)".

3. Delete all smtp server profiles except the default and ensure your settings match below screenshot.
[[File:Smtpsettings.png]]

4. Add a new profile with settings in below screenshot.

TODO - setup backup imap2 container and add settings screenshot here. (old imap2 smtp port 2500)

==Add-ons==

#Install addons: General settings menu (Top right 3 bars) > Addons
#Search and install the addons in below screenshot.
#Configure Mailbox Alert by Right e.g support inbox > Mailbox Alert:
##Check default sound and message.
[[File:Addon2.png]]

==Other General Configurations==

===Configure inbox to display selected columns===

The below options MUST be set in Support Inbox. To do so Right click on a column heading and select the below options:

#Thread
#Starred
#Attachment
#Subject
#Correspondents
#Received: You MUST add column RECEIVED which is the email date received to email inbox because the default Date seems to be the DATE SENT whereas we are primarily interested in the date we received the email. The difference is due to delays in email servers or spam tricks.

The below options MUST NOT be set in Support Inbox

#Junk status: Emails will automatically be moved to Junk folder if the junk icon in this column is clicked. Hence remove Junk Status column so that emails are not accidentally moved to junk. To remove, right-click on a column heading and untick "Junk Status".

===Configure Mail Content===

Preferences > Privacy > untick "Allow remote content in messages"

Support MUST set the "Allow remote content in messages" as blocked in Thunderbird without adding any exceptions for any email id. The reason is remote content i.e embedded images, stylesheets etc is a privacy concern as it sends your private information to the mail sender. We do not know the source of these embedded content in emails so cannot trust them as they can be web bugs.

===Configure Layout View===

Layout view should be set as Vertical view.

===Thunderbird Preferences===

#Preferences > Display > Advanced > untick " Automatically mark messages as read".
#Keep settings as below for Preferences > Security.
#Keep settings as below for Preferences > Composition > Spelling.

[[File:Globaljunk.jpg|500px]]

[[File:Composition.png|500px]]

===Configure MailAlert===

Right click Inbox > Mailbox Alert > Edit Mailbox Alert alerts > Select Default message. Set as below. Once set Right click Inbox > Mailbox Alert > check Default message

[[file:thsh.jpg]]

===Configure Auto Resize Image===

Set the preferences for the Auto Resize Image add-on as shown in the screenshot at the end of this section.

To handle different scenarios when resizing images in emails, follow the below steps:

*Resize all images in the email: Click "Send" followed by "Confirm" on the Auto resize screen.

*Skip resize all images in the email: Click "Send" followed by "Cancel Resizing" on the Auto resize screen.

*Resize Images only in the email trail and skip resizing new images: Click "Resize" on top of your screen and proceed with writing the email/adding new images. When ready to send the email, follow above step for skipping resize all images in the email.

[[File:Resize2.png]]

===Configure Header Tools Lite===

Set the preferences for the Header Tools Lite add-on as shown in the screenshot below.

[[File:headertoolslite.jpg]]

===Configure Dorando Keyconfig===

Menu > Add-ons > Edit Keyconfig Preferences > Disable shortcuts for 'Archive' and 'Junk' as shown in the screenshots below.

[[File:Keyconfig1.png]]

[[File:Keyconfig2.png]]

===Getting access to NEOSYS support calendar===

After confirming that the steps below are successfully done, delete your local personal calendar and make the shared support calendar the default.

In Thunderbird, add a new CalDAV network calendar by navigating to File (Alt+f) -> New -> Calendar.

You will have to enter your nextcloud user and pass.

[[File:Cal1.png]]

Location: https://nextcloud.hosts.neosys.com:44318/remote.php/dav/calendars/support/support_shared_by_steve/

[[File:Cal2.png]]

[[File:Cal3.png]]

It should show a tick if successful, otherwise you will have to Unsubscribe and try again.

[[File:Cal4.png]]

Change the new calendar properties to Refresh every 5 mins.

[[File:Cal5.png]]

===Applying filter for backup failure emails===

In order to filter any backup failure emails as Important, go to Tools > Message Filters and create a new filter with settings as shown below:

Refer to filters list of other staff members.

===Tags===

# Find the prefs.js file in your profile directory:
find ~/.thunderbird/ -name prefs.js
#Edit the prefs.js file
nano /home/gregory/.thunderbird/s3bqjf24.dsfaefawe/prefs.js
#Replace the all the "user_pref("mailnews.tags." with the new set of tag configurations below and restart Thunderbird.
<pre>
user_pref("mailnews.tags.$label1.color", "#FF0000");
user_pref("mailnews.tags.$label1.tag", "Important");
user_pref("mailnews.tags.$label2.color", "#FF9900");
user_pref("mailnews.tags.$label2.tag", "Work");
user_pref("mailnews.tags.$label3.color", "#009900");
user_pref("mailnews.tags.$label3.tag", "Joel");
user_pref("mailnews.tags.$label4.color", "#3333ff");
user_pref("mailnews.tags.$label4.tag", "Greg");
user_pref("mailnews.tags.$label5.color", "#993399");
user_pref("mailnews.tags.$label5.tag", "Arvind");
user_pref("mailnews.tags.$label6.color", "#ff6e6e");
user_pref("mailnews.tags.$label6.tag", "Follow up");
user_pref("mailnews.tags.$label7.color", "#eb007b");
user_pref("mailnews.tags.$label7.tag", "Client issue");
user_pref("mailnews.tags.$label8.color", "#28e2d8");
user_pref("mailnews.tags.$label8.tag", "Delay support");
</pre>

==[[Procedures#Email_Retention_Policy| Email Retention Policy]]==

==[[Procedures#Handling_junk.2Fspam_email| Handling junk/spam emails]]==

==Viewing long running processes on Thunderbird==
When setting up and configuring Thunderbird, using its Activity Manager (Tools > Activity Manager) and possibly using the header bar "Always on top" option on the Activity Manager window may be helpful to know what long running processes are being performed.

[[File:Activity manager.jpg]]

==Deleting Thunderbird accounts==
Check the option "Remove Message Data" when removing accounts from Thunderbird. Otherwise, the old messages stay forever taking space on the disk.

[[File:dt2.png]]

Select Remove Message Data option as shown below.

[[File:dt.png]]

==Clearing out old improperly deleted Thunderbird Accounts==

If you delete Thunderbird accounts without checking the option "Remove Message Data" the old messages stay forever taking space on the disk.

To locate any such unnecessary files

ls -l -h /home/*/.thunderbird/*.default/ImapMail/*/INBOX

<pre>

-rw------- 1 arvind arvind 16M May 25 16:36 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.gmail.com/INBOX
-rw------- 1 arvind arvind 668M Mar 29 08:54 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-1.com/INBOX
-rw------- 1 arvind arvind 242M May 28 14:45 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-2.com/INBOX
-rw------- 1 arvind arvind 755M May 25 12:39 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-3.com/INBOX
-rw------- 1 arvind arvind 420M Apr 24 10:27 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-4.com/INBOX
-rw------- 1 arvind arvind 13G May 28 14:37 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys.com/INBOX

</pre>

Check the dates, which indicate the date of the last message received, to get a rough idea of what accounts are in use and not in use

Check which of the folders are currently in use by looking in Thunderbird settings for each account one by one, "Server Settings", "Local Directory"

Delete the folders and files which are not in use using whatever method you like.

".thunderbird", like all files and folders which start with a dot, is a hidden folder which you can make visible in Nautilus File Manager by pressing Ctrl+h

Configuring Thunderbird

2022-03-14T07:25:31Z

Gregory:

==Setting Up the Inbox Accounts==

The instructions given below explain how to set up the "Support" account on Thunderbird.

1. Open Mozilla Thunderbird. Choose the "Create a new account" option. Click on "Skip this and use my existing email" option on the "Welcome to Thunderbird" window.

[[File:Thunderbird_1.jpg]]

2. In the Mail Account Setup window, enter "support@neosys.com" for Your Name, "support@neosys.com" for Email Address and the password provided by Neosys staff. Click on continue.

[[File:Thunderbird_2.jpg]]

3. Click on "Manual Config".

[[File:Thunderbird_3.jpg]]

4. Key in "imap.neosys.com" for incoming IMAP host name, server hostname "smtp.neosys.com" and port "587" for outgoing SMTP and "support.neosys" as username. For SMTP, change the SSL to "STARTTLS" and authentication to "No authentication". Click on the "Re-test" button and then click "Done".

[[File:Thunderbird_4.jpg]]

BEFORE clicking on inbox and BEFORE adding other inboxes, you MUST refer to [[Configuring_Thunderbird#Adding_email_accounts_with_large_numbers_of_emails | Adding email accounts with large numbers of emails]]. This is because if you add all the accounts together at once, they will all sync together and it will take 1-2 days for the sync to complete.

For backups@neosys.com, support2@neosys.com (Nagios), name.neosys@gmail.com repeat the steps mentioned above, with the below-mentioned replacements.

1. For backups@neosys.com, enter "backups@neosys.com" for Your Name, "backups@neosys.com" for Email address and enter the password provided by Neosys staff.
Enter the username as "backups.neosys".

2. For support2@neosys.com, enter "support2@neosys.com" for Your Name, "support2@neosys.com" for Email address and enter the password provided by Neosys staff.
Enter the username as "support2.neosys".

3. For name.neosys@gmail.com, enter your full name for Your Name, "name.neosys@gmail.com" for Email address and enter the password. You do not need to follow the steps for manual config as the settings will be automatically configured.

[[File:thunderbird_6.jpg]]

[[File:thunderbird_7.jpg]]

==Account Settings==

Verify that the account settings for all the accounts are entered as shown below.

[[File:acsettings1.jpg]]

[[File:acsettings2.jpg]]

[[File:acsettings3.jpg]]

[[File:acsettings4.jpg]]

[[File:Account junk.jpg|500px]]

For backups@neosys.com account, under "Synchronization and Storage", change the setting to "Delete messages more than 365 days old"

The Synchronisation & Storage settings should be changed to the settings shown below only at a convenient time, since downloading all the emails will take a lot of time for accounts with large number of emails. Also see [[Configuring_Thunderbird#Adding_email_accounts_with_large_numbers_of_emails |Adding email accounts with large number of emails]]

[[File:acsettings6.jpg]]

[[File:acsettings7.jpg]]

[[File:acsettings8.jpg]]

===Adding email accounts with large numbers of emails===

After adding email accounts with large numbers of emails, in order to avoid long wait while downloading all the emails, it is best to change the account settings - BEFORE clicking on INBOX, which triggers the download/sync - to download the bodies of only the last 30 days of emails. The email headings are always synced but that does not take long.

See Account Settings, Synchronisation and Storage.

Since searching of emails only works when the bodies of emails have been downloaded, if you need to text search all emails, then, at a convenient time, you can remove or increase the number of days in the setting and somehow trigger the full sync.

[[File:thla.jpg]]

===Configuring Outgoing SMTP Mail Servers===

By default Thunderbird adds a new smtp profile with each Inbox added.

However you only need two, main and backup.

1. Right click on any Inbox > Settings.

2. Scroll down and click "Outgoing Server (SMTP)".

3. Delete all smtp server profiles except the default and ensure your settings match below screenshot.
[[File:Smtpsettings.png]]

4. Add a new profile with settings in below screenshot.

TODO - setup backup imap2 container and add settings screenshot here. (old imap2 smtp port 2500)

==Add-ons==

#Install addons: General settings menu (Top right 3 bars) > Addons
#Search and install the addons in below screenshot.
#Configure Mailbox Alert by Right e.g support inbox > Mailbox Alert:
##Check default sound and message.
[[File:Addon2.png]]

==Other General Configurations==

===Configure inbox to display selected columns===

The below options MUST be set in Support Inbox. To do so Right click on a column heading and select the below options:

#Thread
#Starred
#Attachment
#Subject
#Correspondents
#Received: You MUST add column RECEIVED which is the email date received to email inbox because the default Date seems to be the DATE SENT whereas we are primarily interested in the date we received the email. The difference is due to delays in email servers or spam tricks.

The below options MUST NOT be set in Support Inbox

#Junk status: Emails will automatically be moved to Junk folder if the junk icon in this column is clicked. Hence remove Junk Status column so that emails are not accidentally moved to junk. To remove, right-click on a column heading and untick "Junk Status".

===Configure Mail Content===

Preferences > Privacy > untick "Allow remote content in messages"

Support MUST set the "Allow remote content in messages" as blocked in Thunderbird without adding any exceptions for any email id. The reason is remote content i.e embedded images, stylesheets etc is a privacy concern as it sends your private information to the mail sender. We do not know the source of these embedded content in emails so cannot trust them as they can be web bugs.

===Configure Layout View===

Layout view should be set as Vertical view.

===Thunderbird Preferences===

#Preferences > Display > Advanced > untick " Automatically mark messages as read".
#Keep settings as below for Preferences > Security.
#Keep settings as below for Preferences > Composition > Spelling.

[[File:Globaljunk.jpg|500px]]

[[File:Composition.png|500px]]

===Configure MailAlert===

Right click Inbox > Mailbox Alert > Edit Mailbox Alert alerts > Select Default message. Set as below. Once set Right click Inbox > Mailbox Alert > check Default message

[[file:thsh.jpg]]

===Configure Auto Resize Image===

Set the preferences for the Auto Resize Image add-on as shown in the screenshot at the end of this section.

To handle different scenarios when resizing images in emails, follow the below steps:

*Resize all images in the email: Click "Send" followed by "Confirm" on the Auto resize screen.

*Skip resize all images in the email: Click "Send" followed by "Cancel Resizing" on the Auto resize screen.

*Resize Images only in the email trail and skip resizing new images: Click "Resize" on top of your screen and proceed with writing the email/adding new images. When ready to send the email, follow above step for skipping resize all images in the email.

[[File:Resize2.png]]

===Configure Header Tools Lite===

Set the preferences for the Header Tools Lite add-on as shown in the screenshot below.

[[File:headertoolslite.jpg]]

===Configure Dorando Keyconfig===

Menu > Add-ons > Edit Keyconfig Preferences > Disable shortcuts for 'Archive' and 'Junk' as shown in the screenshots below.

[[File:Keyconfig1.png]]

[[File:Keyconfig2.png]]

===Getting access to NEOSYS support calendar===

After confirming that the steps below are successfully done, delete your local personal calendar and make the shared support calendar the default.

In Thunderbird, add a new CalDAV network calendar by navigating to File (Alt+f) -> New -> Calendar.

You will have to enter your nextcloud user and pass.

[[File:Cal1.png]]

Location: https://nextcloud.hosts.neosys.com:44318/remote.php/dav/calendars/support/support_shared_by_steve/

[[File:Cal2.png]]

[[File:Cal3.png]]

It should show a tick if successful, otherwise you will have to Unsubscribe and try again.

[[File:Cal4.png]]

Change the new calendar properties to Refresh every 5 mins.

[[File:Cal5.png]]

===Applying filter for backup failure emails===

In order to filter any backup failure emails as Important, go to Tools > Message Filters and create a new filter with settings as shown below:

Refer to filters list of other staff members.

===Tags===

find ~/.thunderbird/ -name prefs.js (use the one that looks like this /home/gregory/.thunderbird/s3bqjf24.default/prefs.js)

edit the prefs.js in e.g /home/gregory/.thunderbird/<your profile code>.default/prefs.js

nano /home/gregory/.thunderbird/s3bqjf24.dsfaefawe/prefs.js

Replace the all the "user_pref("mailnews.tags." with the new set of tag configurations below:
<pre>
user_pref("mailnews.tags.$label1.color", "#FF0000");
user_pref("mailnews.tags.$label1.tag", "Important");
user_pref("mailnews.tags.$label2.color", "#FF9900");
user_pref("mailnews.tags.$label2.tag", "Work");
user_pref("mailnews.tags.$label3.color", "#009900");
user_pref("mailnews.tags.$label3.tag", "Joel");
user_pref("mailnews.tags.$label4.color", "#3333ff");
user_pref("mailnews.tags.$label4.tag", "Greg");
user_pref("mailnews.tags.$label5.color", "#993399");
user_pref("mailnews.tags.$label5.tag", "Arvind");
user_pref("mailnews.tags.$label6.color", "#ff6e6e");
user_pref("mailnews.tags.$label6.tag", "Follow up");
user_pref("mailnews.tags.$label7.color", "#eb007b");
user_pref("mailnews.tags.$label7.tag", "Client issue");
user_pref("mailnews.tags.$label8.color", "#28e2d8");
user_pref("mailnews.tags.$label8.tag", "Delay support");
</pre>

==[[Procedures#Email_Retention_Policy| Email Retention Policy]]==

==[[Procedures#Handling_junk.2Fspam_email| Handling junk/spam emails]]==

==Viewing long running processes on Thunderbird==
When setting up and configuring Thunderbird, using its Activity Manager (Tools > Activity Manager) and possibly using the header bar "Always on top" option on the Activity Manager window may be helpful to know what long running processes are being performed.

[[File:Activity manager.jpg]]

==Deleting Thunderbird accounts==
Check the option "Remove Message Data" when removing accounts from Thunderbird. Otherwise, the old messages stay forever taking space on the disk.

[[File:dt2.png]]

Select Remove Message Data option as shown below.

[[File:dt.png]]

==Clearing out old improperly deleted Thunderbird Accounts==

If you delete Thunderbird accounts without checking the option "Remove Message Data" the old messages stay forever taking space on the disk.

To locate any such unnecessary files

ls -l -h /home/*/.thunderbird/*.default/ImapMail/*/INBOX

<pre>

-rw------- 1 arvind arvind 16M May 25 16:36 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.gmail.com/INBOX
-rw------- 1 arvind arvind 668M Mar 29 08:54 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-1.com/INBOX
-rw------- 1 arvind arvind 242M May 28 14:45 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-2.com/INBOX
-rw------- 1 arvind arvind 755M May 25 12:39 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-3.com/INBOX
-rw------- 1 arvind arvind 420M Apr 24 10:27 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys-4.com/INBOX
-rw------- 1 arvind arvind 13G May 28 14:37 /home/arvind/.thunderbird/odi5mrka.default/ImapMail/imap.neosys.com/INBOX

</pre>

Check the dates, which indicate the date of the last message received, to get a rough idea of what accounts are in use and not in use

Check which of the folders are currently in use by looking in Thunderbird settings for each account one by one, "Server Settings", "Local Directory"

Delete the folders and files which are not in use using whatever method you like.

".thunderbird", like all files and folders which start with a dot, is a hidden folder which you can make visible in Nautilus File Manager by pressing Ctrl+h

EXODUS Knowledge

2022-03-07T10:43:24Z

Gregory: /* Speedup postgresql importing databases */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries==
Dictionaries, the files used to describe the fields of a file's record.
Unlike in AREV, there is a copy of all dictionaries in each pgsql database (In AREV, updating a dictionary would affect all the databases).

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Updating a pgsql function in an exodus dictionary==

PENDING

==Development and deployment using 'dat' files==

===Rationale===

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

==='dat' files===

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

===Location of dat files===

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Editing and deploying a 'dat' file===

It is currently a three step process to edit and deploy such 'dat' files.

====Edit the 'dat' file====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

====Copy all 'dat' files to ~/dat ====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

====Copy all programs and 'dat' files to ~/live/bin|lib|dat====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

==Upgrading Exodus==

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
or reload postgres may also work

EXODUS Knowledge

2022-03-02T08:08:40Z

Gregory:

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries==
Dictionaries, the files used to describe the fields of a file's record.
Unlike in AREV, there is a copy of all dictionaries in each pgsql database (In AREV, updating a dictionary would affect all the databases).

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Updating a pgsql function in an exodus dictionary==

PENDING

==Development and deployment using 'dat' files==

===Rationale===

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

==='dat' files===

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

===Location of dat files===

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Editing and deploying a 'dat' file===

It is currently a three step process to edit and deploy such 'dat' files.

====Edit the 'dat' file====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

====Copy all 'dat' files to ~/dat ====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

====Copy all programs and 'dat' files to ~/live/bin|lib|dat====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

==Upgrading Exodus==

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==
For Ptcy.

#Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

==Speedup postgresql importing databases==

Can be sped up by a factor of 10 BUT may result in corrupt files in case of hard OS crash where the dirty memory is not flushed to storage.

Therefore it is advisable to use this feature only briefly and REMEMBER TO REMOVE IT!

Email to support@neosys.com whenever using it or use personal notes not memory.

nano /etc/postgresql/12/main/postgresql.conf

Uncomment or add a line
fsync = off
Restart postgres … will break any neosys/exodus running processes.
r postgresql
OR
service postgresql reload

EXODUS Knowledge

2022-02-07T08:30:12Z

Gregory: /* General */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries==
Dictionaries, the files used to describe the fields of a file's record.
Unlike in AREV, there is a copy of all dictionaries in each pgsql database (In AREV, updating a dictionary would affect all the databases).

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====

~/neosys/doall [LIVE|TEST] <DBCODE|ALL> < ACTION [OPTIONS,..] | bash -- COMMAND ]

Example:
./doall TEST ALL create_site t-
Result:
Create site for ALL TEST databases, using domain name prefix option "t-".

Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Updating a pgsql function in an exodus dictionary==

PENDING

==Development and deployment using 'dat' files==

===Rationale===

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

==='dat' files===

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

===Location of dat files===

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Editing and deploying a 'dat' file===

It is currently a three step process to edit and deploy such 'dat' files.

====Edit the 'dat' file====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

====Copy all 'dat' files to ~/dat ====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

====Copy all programs and 'dat' files to ~/live/bin|lib|dat====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

==Upgrading Exodus==

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==

Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

EXODUS Knowledge

2022-02-07T08:25:26Z

Gregory: /* Process Hung or consuming 100%+ CPU */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries==
Dictionaries, the files used to describe the fields of a file's record.
Unlike in AREV, there is a copy of all dictionaries in each pgsql database (In AREV, updating a dictionary would affect all the databases).

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming +100% CPU===
If process is consuming +100% CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit +100% break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====
Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Updating a pgsql function in an exodus dictionary==

PENDING

==Development and deployment using 'dat' files==

===Rationale===

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

==='dat' files===

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

===Location of dat files===

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Editing and deploying a 'dat' file===

It is currently a three step process to edit and deploy such 'dat' files.

====Edit the 'dat' file====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

====Copy all 'dat' files to ~/dat ====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

====Copy all programs and 'dat' files to ~/live/bin|lib|dat====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

==Upgrading Exodus==

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==

Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

EXODUS Knowledge

2022-02-07T08:24:45Z

Gregory: /* Process Hung or consuming 100%+ CPU */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries==
Dictionaries, the files used to describe the fields of a file's record.
Unlike in AREV, there is a copy of all dictionaries in each pgsql database (In AREV, updating a dictionary would affect all the databases).

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming 100%+ CPU===
If process is consuming 100%+ CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.

A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit 100%+ break in test debugger.

If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.

Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

==./doall==
====General====
Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Updating a pgsql function in an exodus dictionary==

PENDING

==Development and deployment using 'dat' files==

===Rationale===

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

==='dat' files===

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

===Location of dat files===

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Editing and deploying a 'dat' file===

It is currently a three step process to edit and deploy such 'dat' files.

====Edit the 'dat' file====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

====Copy all 'dat' files to ~/dat ====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

====Copy all programs and 'dat' files to ~/live/bin|lib|dat====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

==Upgrading Exodus==

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==

Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running

EXODUS Knowledge

2022-02-07T08:24:03Z

Gregory: /* Debugging Exodus */

==TMUX Screens==

To create the EXODUS maintenance/programming environment
exodus#: ./tmux.exodus

===Screens===
<pre>
0: /
Work regarding exodus server. i.e /etc/nagios or general networking or /var/log/mail.log

1: /root/exodus/
Exodus core is installed Contains build and make libraries here ./libexodus

2: /root/exodus/libexodus/exodus
Core exodus (libraries that emulate the AREV system e.g mv.h )

3: /root/exodus/cli/src
Exodus Command Line Interface e.g edir, edic, list, listfiles, delete, createdb

4: root/exodus/service
EXODUS core default host directory but also contains scripts that create/manage requirements of a service (create_service, create_db, create_site, backup_db, restore_db, start, stop, status)
One exception is NEOSYS www files are in ./www, which pulled from NEOSYS www git.

5: /root/exodus/service/src
Exodus core service programs (programs that are not related to NEOSYS agency operations and can be used to support other non-NEOSYS services) eg listen, usersubs, sendmail, openfile, select, sort, list,.

Extra info:
sql directory contains basic dictiorary files converted to sql instructions used when creating a EXODUS psql database
e.g dict_voc.sql is a file that describes what to expect in certain fields of the voc file.
TODO: compall currently throws few warnings. Read them and familiarise. In case new warnings appear, notify Steve.

6: /root/neosys
NEOSYS specific client installation management scripts e.g .doall, import_db and import_files (from AREV), ./run ./test (see below for these two commands)

7: /root/neosys/src
NEOSYS Agency programs (see section How AREV programs are distributed in EXODUS src/*.cpp
../ has .git

8: /root/hosts
client specific files e.g logs, images, (shared ./www -> root/exodus/service/www

9: /root/exodus/test/src
series of programs that test whether subroutines (specifically Exodus core libraries) work as they should eg for the var lib - "assert( var("11111").isnum());"

10: bash /bin/top
Monitor for NEOSYS client processes
Displays customised format: sorted by total time spend and by processor time.
Press "=" : to change to standard top format
Press shift + w : to save format settings, replacing the old customised top format
To go back to the customised format do:
Press "o"
Type "COMMAND=serve_agy"
shift + t : this sorts by total time spent
shift + p : this sorts by processor time
shift + w : to save this setting and enter 'y' to confirm

11: /var/log/syslog
Log of NEOSYS client requests

12: /root/exodus/service/quick
Displays the last time the svr file (in /root/hosts/<client>/data/<dbname>) was updated.
</pre>

==Object Code/Libraries==
LIVE and TEST processes use different sets of object code.
TEST processes use libraries in ~/lib/, whereas LIVE processes use object code in ~/neo/lib

This means development & testing can be done stress free on TEST database, as opposed to testing on production databases.

When compiling using edic, the TEST object code is updated if the compilation is successful. (~/lib)
In order to apply a tested patch to LIVE see [[Update LIVE programs]].

==Dictionaries==
Dictionaries, the files used to describe the fields of a file's record.
Unlike in AREV, there is a copy of all dictionaries in each pgsql database (In AREV, updating a dictionary would affect all the databases).

==Processes==
The TEST process for all database use the same object code stored in /root/lib, whereas all LIVE process use the object code in /root/neo/.

==Postgres==
Connect into postgres shell:
alias psql='sudo -u postgres psql'

Delete a database:
sudo -u postgres dropdb <dbcode>

==Debugging Exodus==

===Debugging Live/Test services===

./test <dbname> - execute test programs on _test database.

./run <dbname> - execute test programs on live database.

===Debugging var values===

Explanation of var flag value in gdb debug mode.

<pre>
(gdb) p varx
$2 = {var_str = "", var_int = 0, var_dbl = 0, var_typ = {flags_ = 2}}
</pre>

Each exodus var object contains 3 variables of type string, int and double.

All 3 may be unassigned or only the string be assigned or just the int and double e.t.

You can tell which variable type of a var object are set based on the var's flag value.

1 = Means the string is assigned
2 = Means the int is assigned
4 = Means the double is assigned

Example: If the flag is 7 (1+2+4), then all types are assigned.

===Process Hung or consuming 100%+ CPU===
If process is consuming 100%+ CPU then it is likely stuck in an infinite loop.

./doall <CLIENT> stop

./run <CLIENT>

./test <CLIENT>

Search through recent logs for <CLIENT> looking for requests that didn't return OK at the time nagios alerted of Hung process.
A user may also instigate the error again, so monitor the CLIENT's service CPU usage.

If cause found recreate it in test and once test process hit 100%+ break in test debugger.
If cause not found, you will have to continue checking logs for the request that caused it or hope a user recreates the issue in live debugged service.

Once you've broken in the debugged with the error, enter "n" and hold the enter key until you just seeing line repeat.
Pay attention to start of loops code. E.g WHILE or FOR. (there is likely an error is there)

#

==./doall==
====General====
Screen 6: ./doall script contains all the necessary information(codes) to setup an installation.
It includes scripts to backup, restore, create an Apache site, create/start/stop/status a service, import an AREV database into postgres and more.

====backup_db====

*Does a backup & restore of a LIVE database into the corresponding TEST database.
*Backup <dbcode>.sql file is written to /root/backups/sql; which is rsynced to nl19:/backups/current/exodus/
*Unlike AREV, postgres can perform a "backup" of a database whilst the system is in use.

==Git==

There are two repositories, one for EXODUS and the other for NEOSYS.

===Using git to make changes===

Before following steps you must have a tested updated to a program/file/script. Do not commit untested changes to avoid a messy git history of reverts.

Update your local repo before committing to local repo using the g alias for "git pull --ff-only ; git push && git status'":
<pre>g</pre>
Check which updates/files have not yet been staged and/or committed:
Add your updates to the staged area:
<pre>git add <filename></pre>
or if all the changes made need to be staged:
<pre>git add -a</pre>
Make a commit with a descriptive message on purpose of updates:
<pre>git commit -m <description n purpose of changes></pre>
Again use g alias:
<pre>g</pre>

===Other useful git cmds===

Do not use this commands unless you know what WILL happen.
*git pull - Instead use the safe "git pull --ff-only"
Stick to the alias "g" which does "git pull --ff-only ; git push && git status'"

*git log - display history of commits to master branch
*git diff - display the differences in files between working files and files in local repo.
*git status - display: which updated files are staged/not staged(tracked)
*git stash -
*git branch - switch to a new branch
*git branch <branchname> - will switch to this branch only if it exists
*git checkout - DESTROYS/Updates the state of local working directory with the state of the files in the local repo. (YOU WILL LOSE non staged updates)
*git restore <filename> - DESTROYS/Updates the file with the version in the local repo. (like with git checkout but for a specific file)
*git checkin -
*git revert <commitHash> - reverses a specific commit (use git log to get the chosen commit hash)
*git ..

==Converting AREV programs to EXODUS==
===Decompile AREV to C++===

Do in Master AREV Installation Maintenance mode: (win7)

#ATTACH ADECOMC
#*ADECOM <programname> *single program
#*ADECOM <prog1> <prog2>
#*ADECOMALL *all programs (CHECK FIRST)

Include the option "(V)" in the command to print the C++ to a notepad, which can easily be copy and pasted.

===Getting C++ program to Exodus Installation===

Two methods: (use the first for one off programs)

====Copy & Paste====

Compile program and open source code in notepad

ADECOM <programname> (V)

Find which directory the program belongs. e.g fin/med/job/sys e.t

find /cygdrive/d/exodus/pickos | grep <program name>

If cpp in SYS, the program belongs in ~/exodus/service/src OR if in MED JOB FIN GEN AGY the program belongs in ~/neosys/src

Copy the source code in the text file from the first command.

In Exodus, create a new .cpp (in correct directory) and paste the source code.

Compile & Test

====Rsync====

/d/exodus/arev/syncup.sh

If cpp in SYS then: ~/exodus/service/src ./getpickos

If cpp in MED JOB FIN GEN AGY then: *~/neosys/src

===Compile C++ files to TEST system===

#*./test <DBNAME>
#*~/neosys ./doall TEST <DBNAME> restart #to get one service to start start using the new lib files
#*~/neosys ./doall TEST all restart #to get all the services to start start using the new lib files

===Install C++ files to LIVE System===

#~/exodus/service/ ./copyall #to copy all the ~/lib and bin files to ~/live/lib and bin ... which is used by all exodus/live services

==Writing Standard Exodus Core Function/Method Testing==
Screen 9: ~/exodus/test/src/
There are a series of test programs that check whether methods/functions behave as intended.
They do this using the function, assert.. a 1 or more argument values produce one and only one output)

e.g test_multilang.cpp or test_sort.cpp

Two methods of running test programs:
*Screen 9: make test
*after compiling using edic/compile/c, enter test_prog_name. (Since compile has moved it to ~/bin)

Difference between the two methods is make calls gdb directly;
whereas ~/bin/test_prog_name uses exodus compile program
#~/neosys ./doall LIVE all restart

==Updating a pgsql function in an exodus dictionary==

PENDING

==Development and deployment using 'dat' files==

===Rationale===

Part of system development is the creation of various data that is neither programs nor layout i.e. not cpp, h, html, js, php files etc.

For example:

*Dictionaries are data about data.
*Language files are data about text to use for various languages.
*“Change logs” are data about changes in the system.

Historically, in EXODUS and NEOSYS, the above data has been deployed in exodus database files using SQL text files. However SQL files are not convenient for development.

Therefore, 'dat' text files will be used now so that standard development tools including editors and git can be fully exploited.

==='dat' files===

Each database file is represented by an os directory of the same name.

Each record in the database file is represented by an os text file where filename is the primary key.

For example a record with key '''DEADLINE''' in a dat file '''dict.materials''' would be represented as an os text file '''dat/dict.materials/DEADLINE'''

Each line in the os text file represents one field in the db record. In other words, db record FM characters are represented by new line characters in 'dat' files. Any actual new line characters required in the record, and any backslashes, are escaped and appear as '\n' and '\\' in 'dat' files. Other database field separator characters such as VM, SM, TM and STM are stored without any conversion.

===Location of dat files===

The development versions are stored in exodus and neosys src/dat dirs. They form part of the standard git repositories in parallel with cpp files.

The operational versions are stored in ~/dat and ~/live/dat alongside bin and lib dirs and are automatically installed into databases as database files on service startup. Any database functions embedded in the text files (pgsql) are also automatically installed at the same time.

===Editing and deploying a 'dat' file===

It is currently a three step process to edit and deploy such 'dat' files.

====Edit the 'dat' file====

Note that EXODUS service and NEOSYS service have different src/dat folders.

Editing language items:

edir dat/alanguage/SCHEDULES*ARABIC

Editing a dictionary item:

edir dat/dict.materials/DEADLINE

Editing a pgsql function in a dictionary:

edir dat/dict.materials/DEADLINE 8

====Copy all 'dat' files to ~/dat ====

This step might be removed at a later date.

This will cause all test databases to immediately restart and load any 'dat' file changes into dictionaries and data files and also create any new or modified pgsql functions.

If any ~/neosys/src/dat files were edited:

cd ~/neosys/src

./compall dat

and/or, if exodus/service/dat files were edited

cd ~/exodus/service/src

./compall dat

====Copy all programs and 'dat' files to ~/live/bin|lib|dat====

This should only be run after testing. It will cause all live databases to automatically restart and do the same as the test databases mentioned above.

cd ~/exodus/service

./copyall CONFIRM

==Upgrading Exodus==

#Inform clients.
#*Client hosted - Schedule a planned upgrade with client.
#*NEOSYS hosted - Inform all clients/users of planned morning upgrade, 24hours in advanced. (TODO: 'What's New in NEOSYS') AND (emailallusers.cpp)
#If the morning's USB backup failed, do NOT continue upgrade procedure because if upgrade fails, we need a way to roll back system to last working state.
#Check upgrade works on test service using:/root/neosys/upgrade.all.sh test
#Check the latest patch is working.
#Run/root/neosys/upgrade.all.sh live
#Inform users their system has been upgraded and they can login.

===Roll Back Upgrade===

If for any reason the entire upgrade needs to be reverted (excludes clients database), follow steps below:

#Inform users that there is a problem and services need to be stopped.
#Stop all exodus processes.
#Restore the morning's USB backup by: (Note this backup contains: source/obj code, /etc, neosys scripts but not client data)
##Login into container's host and rsync the morning's usb backup to the container's /:
#*<pre>rsync --dry-run -avz -e 'ssh -p <C_SSHPORT>' /backups/usb/<C_CODE> <C_CODE>:/ (Example: rsync --dry-run -avz -e 'ssh -p 19582' /backups/usb/ad1 ad1:/</pre>)
#Quick sane check, the latest git commit is not the latest git commit in the remote repo.
#Inform users they can login.

==Configuring services to autostart during login==

Create a file 'autostart.cfg' in the data folder

touch /root/hosts/<dbname>/data/autostart.cfg

This will autostart a database service as long as at least one database service in that directory is running.

For example, demo service will autostart during login if,
#/root/hosts/demo/data/autostart.cfg exists and
#demo_test service is running