Updated: Version 3.5

Remote synchronisation

mSupply's remote synchronisation allows stores running mSupply to send data to a central mSupply server for combined reporting.

It's like this:

Definitions

  • The primary server is the central mSupply server, which controls the master system data which is common across all stores e.g. names, items etc.
    • In any given setup, there will only be one primary server, and it must be running a web server.
  • A satellite server is a regional mSupply server (or single-user standalone copy of mSupply) which connects to the primary server when an internet connection between them is available.
    • The syncing process is controlled from the satellite server, so the satellite server doesn't need to be running a web server for syncing.
  • The primary server and all of the satellite servers are sync sites.
  • Each sync site has a unique ID and connection parameters (IP address, username and password), setup in the synchronisation preferences, along with a list of the other sync sites with which it can send/receive synchronisation data. These settings are complicated, easy to get wrong, and so should only be modified by Sustainable Solutions.
  • To preserve data integrity, and to avoid potential clashes where more than one sync site tries to modify the same data, only one sync site can edit/update any specific type of data.
    • System data which is common to all stores (e.g. suppliers, customers, items) can only be edited or imported on the primary server - changes are synced to all satellite servers (see below for details).
    • Store-specific data (e.g. transactions, locations, abbreviations) can only be edited or imported on a sync site. Any changes to store-specific data are only synced to the instance of the corresponding store on other sync site(s), depending on the store sync preferences (see below for details).

Stores to be synced will exist as separate instances of the same store on more than one sync site (one of which will be the primary server), but store-specific data, and the core data for the store itself (i.e. not including store preferences), can only be edited on one of these sync sites - the site on which the store is defined as Active. This is controlled by the store preferences on the Synchronisation tab for each virtual store on each sync site:

Synchronisation type drop down list: Shows the sync type of the current store. Will be editable if you have unlocked it using the Click to unlock button.

Click to unlock button: Click it to enter the unlocking password to enable you to edit the settings on this tab.

Set as local store checkbox: Will be editable if you have entered the password using the Click to unlock button and the store's Synchronisation type is set to Active. If checked, this will change the store's Sync ID to be that of the current site and will uncheck any of the elements in the Local column in the Sync with table.

Include prescriptions in sync checkbox: Only appears when the Synchronisation type is set to Active. If checked, this will also sync prescriptions and related dispensary data. As this can potentially generate a lot of extra sync traffic, the default is that it is switched off.

Sync ID field: Displays the sync ID of the current store so that you can see which one in the table below you're talking about! The store's sync ID should almost always be the ID of the site where the store is active.

Sync with table: Shows the other sites in your sync setup that the current store will sync with. Also defines what relationship those other store instances have to your current one and therefore what records need to be sent to that site from the instance of the current store on the current site. Will be editable if you have entered the password using the Click to unlock button. Note that the Local column is only editable if the Synchronisation type is also set to Active. Checking any checkbox in the Local column will set the store's Sync ID to be that of the selected row, and will uncheck any others as well as the Set as local store checkbox.

This setting is a bit confusing, but it does work to understand it as 'whatever the sync type of the store is on that server'.

If your version of mSupply server is pre 3.50, after changing sync settings for a store you will have to restart the mSupply application on the primary server for the changes to be applied.

  • Once a store has been set up (see the relevant parameters below), item and store visibility for the store needs to be set up on the primary server - see Virtual stores, Controlling item visibility.
  • To do that, once you log in to the primary server, you need to switch to the new store - see Switching store
  • To do that, you need to have permission to log in to it - see Managing users (It's a good thing this doesn't happen often! :-D)

These settings are necessarily complex and should only be modified by Sustainable Solutions, after discussions to agree on the configuration which is the best fit for requirements.

Store sync types

Each store in a synchronised system needs to have a sync “type” as described here.

Active

A store whose sync type is Active means that the store's site-specific data can be edited (added, edited, deleted) on the sync site where you are editing/viewing this store.

Collector

A store whose sync type is Collector means that the store's site-specific data can not be edited on this sync site. The store on this server receives all store data from an active store on another sync site. i.e. it is a replica of a corresponding active store on another sync site. This means that the store can be examined and reports generated on this sync site.

Transfer

A store whose sync type is Transfer is similar to the Collector type except that the store on this sync site is used only as a vehicle for receiving stock transfers or requisitions from other stores on the same sync site and passing them on to the corresponding store on other sync sites. It doesn't receive any other store data (such as stock lines). That is, it is not a replica of an active store, and should not be used as such. The store can not be examined or have reports generated on this sync site. The benefit of a Transfer store over a Collector store is that Transfer stores require less data to be synced between servers.

None

A store whose sync type is None means that the store's site-specific data can be edited, but that data is not synced with any other sync site. The only exception is for the store record itself on a satellite, which will be synced to the primary server (so that the primary has a copy of all stores and can control visibility). In that case, the instance of that store on the primary should have a sync type of Transfer.

Even if a sync store has sync type = none, it can still receive incoming sync records from outside. These incoming records will trigger the code to see if the records need to be forwarded on elsewhere.

Store sync-with options

The Store 'sync-with' options control which store-specific synchronisation data gets sent to the other sync sites which are linked to that sync site:

None

A value of None means that this store doesn't sync with the corresponding store on the selected sync site. That is, there is no instance of this store on that sync site.

Active/collector

A value of Active/collector means that store-specific data edits, including any edits to the core store data, are synced to the corresponding store instance on the selected sync site. The store instance on the selected sync site will normally be of type Active or Collector.

Transfer

A value of Transfer means that any updates to the core store data, or any stock transfers or requisitions into that store, are synced to the corresponding store instance on the selected sync site. No other transactions or other store-specific data will be transferred. The store instance on the selected sync site will normally be of type Transfer.

For example:

Store A

An example of a store on the primary server which needs to also be reportable on another satellite server.

Store A exists as an Active store on primary server and as a Collector store on satellite server 1:

  • On primary server store-specific data for this store can be entered. This data is then synced to satellite server 1.
  • On satellite server 1 store-specific data for this store cannot be entered. The store can receive stock transfers or requisitions from other stores on the same sync site (satellite server 1). These will be synced to primary server.

Store B

An example of a store on one satellite server which needs to receive stock transfers from a store on another satellite server.

Store B exists as a Collector store on primary server, an Active store on satellite server 1, and a Transfer store on satellite server 2:

  • On satellite server 1, store-specific data for this store can be entered. This data is then synced to primary server.
  • On primary server, store-specific data for this store cannot be entered. The store can receive stock transfers or requisitions from other stores on the same sync site (primary server). These are then synced to satellite server 1.
  • On satellite server 2, store-specific data for this store cannot be entered, and is not synced from primary server. It can receive stock transfers or requisitions from other stores on the same sync site (satellite server 2). These are then synced to primary server, and then to satellite server 1.

Store C

Another example of a store on one satellite server which needs to receive stock transfers from a store on another satellite server.

Store C exists as a Collector store on primary server, an Active store on satellite server 2, and a Transfer store on satellite server 1:

  • On satellite server 2, store-specific data for this store can be entered. This data is then synced to primary server.
  • On primary server, store-specific data for this store cannot be entered. It can receive stock transfers or requisitions from other stores on the same sync site (primary server). These are then synced to satellite server 2.
  • On satellite server 1, store-specific data for this store cannot be entered. It can receive stock transfers or requisitions from other stores on the same sync site (satellite server 1). These are then synced to primary server, and then to satellite server 2.

Store D

An example of a store on a satellite server which needs to also be reportable on the primary server.

Store D exists as an Active store on satellite server 2 and as a Collector store on primary server:

  • On satellite server 2, store-specific data for this store can be entered. This data is then synced to primary server.
  • On primary server, store-specific data for this store cannot be entered. It can receive stock transfers or requisitions from other stores on the same sync site (primary server). These are then synced to satellite server 2.

Store E

An example of a store which is local to the primary server only.

Store E exists only on primary server:

  • On primary server, store-specific data for this store can be entered. This data is not synced anywhere.

Store F

An example of a store which is local to a single satellite server

Store F exists on satellite server 1 as type 'None' but it must also exist on the Primary server (as type 'Transfer' so that it can be managed (e.g. visibility):

  • On satellite server 1, store-specific data for this store can be entered. This data is not synced anywhere.

Server configuration settings in the graphic below are incorrect!

The table below defines which data types are considered as system or store-specific. Other types of data (e.g. preferences, user permissions are always local to the sync site, and are not synchronised).

Data Sync records Notes
Items System Including item-related data e.g. item categories, units
Names (except patients) System Including name-related data e.g. name categories, contacts
Visibility of items and names System
Merging of items and names System
Groups and departments System
Item master lists System
Budgets and accounts System
Transaction categories System
Purchase order categories System
Location types System
Customer stock history and requisitions Store
Locations Store
Stock and stocktakes Store
Transactions Store Including other transaction-related data e.g. inventory adjustments, backorders, builds, but not prescriptions
Purchase Orders Store Unless the Procurement is done centrally preference is switched on, in which case purchase orders can be entered on the primary for any store, regardless of its sync type
Goods received Store
Patients Store Including other patient-related data e.g. medication records
Prescribers Store
Prescriptions Store Only if an additional store preference is turned on (off by default)

Stores

These are a special case. The rule for these is that, if the store is an Active store, then any changes to the store record itself (anything on the General tab) will be synced to the primary server, and then propagate on to any other sync sites as determined by the “syncs with” settings for that store on the primary server (i.e. to any other sync sites where “syncs with” is not None). A store should only ever be an Active store on one sync site at a time.

The Set as local store checkbox is only editable for an Active store. This is used internally, and is set when the store is originally created in order to record which sync site it was created on. It should not normally be changed.

Note that store preferences and sync settings are always local to the sync site and are not synchronised.

Centralised procurement

If this preference is switched on in the primary site, then purchase orders can be entered for any store on the primary, regardless of its sync type (i.e. not just for active stores).

If it is switched on in a satellite, then purchase orders cannot be entered for any store on the satellite, even if it is active.

When this preference is changed on the primary, the change will propagate to all the satellites as well.

It is still possible to edit the preference on the satellite (e.g. to allow local procurement for stores active on that site), but if that is the case, then the related permissions for purchase orders for those stores on the primary will need to be disabled manually in order to prevent purchase orders for those stores being editable on both sites.

Dispensary data

Prescriptions

By default, prescriptions and any other operations in dispensary mode which affect stock levels are not synced to the primary server, unless the Include prescriptions in sync option is enabled in the store sync preferences, as these can potentially generate a lot of sync traffic and there is usually no need for the central users to have this level of detail - the satellite will usually have another store which supplies the dispensary and its transactions will be synced to the primary.

If this preference is switched off in a dispensary store on a satellite, the following data will not be synced back to the primary:

  • Transactions (including backorders, builds etc.)
  • Prescriptions
  • Stock and stocktakes

Patients

Patients in mSupply are a special kind of customer, but for the purposes of synchronisation, we need to treat them differently - as store-specific data rather than system data. This is because patient data only needs to be shared with other dispensing sites.

For an Active store which is also a dispensary, patient records will be synced to the primary server. Newly created patients will also be made visible in any stores on the primary server which have been set to dispensary mode. Subsequently, patient visibility is controlled from the primary server in the same way as other name (customer & supplier) records.

At the moment, patient records are not shared with other dispensary satellites although this may change in future e.g. if there is a country-wide patient registration system, in which case we would have to assign a store to each patient in order to control where it can be edited. Otherwise, if the same patient record existed on more than one dispensary satellite, then either satellite could update it and that would update the copy on the primary server, but the copy on the other satellite would remain untouched.

Patients are not editable on the primary server unless all dispensary stores are local to the server itself (i.e. there are no dispensary stores active on any of the satellite sites).

Other Dispensary Data

Prescribers, abbreviations, patient medication records, and repeats are also treated as a special kind of store-specific data, similarly to patients i.e. they are synced from an active dispensary store on a satellite back to the primary server, but not shared with other satellite dispensary stores, and not editable on the primary server unless all dispensary stores are local to the server.

Transfers occur when there are two stores involved in a transaction, and includes stock transfers, requisitions (from a mobile store) and internal requisitions (from another desktop store). In a syncing system, very often these two stores are not active on the same site, so there has to be extra processing to ensure that both halves of the transaction are synced to both the initiating store/site and the responding store/site.

Both stores need to exist on both sites, and usually the initiating store is a transfer store on the responding site, and vice-versa. As of mSupply v3.83, unless both stores are active on the same site (in which case, everything can be done locally), the bulk of the processing is done on the central primary server when it detects the initiating half of a transfer transaction. In simple terms, the logic is something like this:

  1. when the primary detects the initiating half of a transfer transaction
    1. it creates the responding half of the transaction, but with a dummy invoice/serial number of -1
    2. it ensures that both halves of the transaction are synced to both initiating and responding sites
  2. when the responding site receives the responding half of the transaction
    1. it assigns the next invoice/serial number for the store and sends that back to the primary
    2. it creates a log message for the initiating half of the transaction and sends that back to the initiating site via the primary
  3. any subsequent changes to the initiating half of the transaction (usually very limited in scope) on the initiating store/site will be synced back to the primary (according to the normal sync rules)
  4. any subsequent changes to the responding half of the transaction on the responding store/site will be synced back to the primary (according to the normal sync rules)

Stock transfers

  • This is where the customer in a customer invoice is another store
  • The initiating half of the transaction is where the customer invoice is finalised in the initiating store/site
  • The responding half of the transaction is a supplier invoice (on hold by default) in the responding store/site

Mobile Requisitions

  • This is where the supplier in a supplier (aka request) requisition is another store
  • The initiating half of the transaction is where the supplier requisition is finalised on mobile
  • The responding half of the transaction is a response requisition in the responding store/site
    • this shows up as a customer requisition on mobile if the responding store is active on another mobile site
    • or as a response requisition if the responding store is active on a desktop site
      • the user can create one or more customer invoices to fulfil the requistion
      • these customer invoices (when finalised) will generate corresponding supplier invoices back in the initiating store as stock transfers

Internal Requisitions

  • This is where the supplier in a purchase order is another store, and is a two-stage process
  • The initiating half of the first transaction is where the purchase order is confirmed in the purchase order store/site (i.e. where the purchase order is editable)
    • if centralised procurement is enabled, this will be the primary site, otherwise it will be the normal initiating store/site (i.e. where the purchase order's store is active)
  • The responding half of the first transaction is a customer invoice in the responding store/site
  • The initiating half of the second transaction is where the customer invoice is finalised in the responding store/site
    • note that adding extra customer invoice lines before the invoice is finalised will create corresponding new goods received lines
  • The responding half of the second transaction is a goods received record in the initiating store/site (i.e. where the purchase order's store is active)
  • If centralised procurement is enabled:
    • the received quantities for the original purchase order lines will be updated on the primary whenever the corresponding goods received lines are received there (and forwarded from there to the initiating store/site)
    • when the primary receives any subsquent updates to goods received lines from the initiating store/site, it will update the quantities in the corresponding purchase order lines
  • If centralised procurement is not enabled, the received quantities for the original purchase order lines will be updated in the initiating store/site when it receives the goods received lines from the primary
  • On the primary server if you login in supervisor mode, you can then run reports on one or more stores. The reports can answer questions such as:
    • How much stock on hand of item X (or all items) are there at each location?
    • What is the value of stock on hand across the whole system?
    • How many of item X is being used each month at each location?
  • When the synchronisation system is turned on, a new Special > Users report… > Sync report… menu item is available. Choosing this will create a report showing the last time each of your sync stores connected to the primary server. Those that connected more than a month ago will be highlighted in red.
  • A license from Sustainable Solutions to use the web server module and the synchronisation module on the primary server.
  • Internet at each sync site. This doesn't have to be on all the time for each satellite server, but at least an hour or so per day or per week (depending on the transaction volume and the speed and latency of the internet, and how often you need primary server data updated). Obviously, the internet needs to be on at the primary server at the same time.

On a satellite server

If there are queued sync records, there will be a message at the bottom left of the navigator which shows the number of sync records in the queue.

On the primary server

  • Choose Help > About mSupply
  • At the bottom left of the window there is a list of tables and the number of records in each table.
  • Scroll the list to near the bottom, and you will see the number of records in the sync_out table.
    • If the number is zero, your copy of mSupply is up to date
    • If the number is growing from day to day, there are possible reasons:
      • you need to provide more internet time or faster internet
      • there may be a problem that needs the attention of Sustainable Solutions. Before contacting us, make sure the internet is connected for an hour, and see if the number is decreasing or not.

Don't try this at home unless you are very confident in your abilities!

This example is for a simple sync system with some satellites where data is entered, one store on each satellite, and a primary server that is just used to aggregate data, set up core data, and run reports.

  1. Create new data files
  2. on primary;
    1. choose prefs > sync
    2. enter “1” for sync ID
    3. enter remote sites into the syncs with list. The name is only for identification of the site
    4. Choose file > edit users
      1. create a user like xsyncuser
      2. note the password,
      3. xsyncuser must have a default store set
  3. If stores are to be created on satellite, then: on each satellite site (else if creating on primary, skip to below)
    1. Choose prefs > sync
    2. enter site’s own ID in “sync ID” field (same ID set on primary)
    3. Enter 1 entry in “sync with” section
    4. ID is the ID of the primary site
    5. user & pass: as for user set up on primary
    6. IP address of primary (include :port)
    7. Choose Special > stores
      1. There will be one store created by default with the data file (General)
      2. View (double-click) the General store, go to > sync tab
      3. set to active store
      4. set to local store
      5. set the type for the row in the list for syncing with the primary to “active/collector store”
      6. Rename the store to the name you want (e.g. My Site General store)
      7. Click OK to Save the record- it will show in the sync queue in the Navigator, and will sync up to the primary.
  4. Back on the primary:
    1. Open the store that has just synced :
      1. Set to be a collector
      2. Set sync with for satellite ID to be active/collector
      3. The Sync ID field should show the ID of the remote site
      4. On the visibility tab set item and name visibility as needed.
    2. Choose prefs > sync
      1. double click the site in question. Click generate sync records for site
        1. Choose the option- update existing store
        2. Records will be generated for items, names etc and will sync to the satellite
  5. Repeat step 3 on each Satellite and Step 4 on the Primary after setting up each Satellite
  1. If stores are to be created on primary, then:
    1. create stores
    2. set item and name visibility as needed
    3. in File > Prefs
      1. on sync tab, add a record for each site, assigning a sync id and a name (it's not really “user name” - more “site name”)
    4. in Special > Stores
      1. for each store
      2. Open the store that has just set up in prefs :
        1. Set to be a collector
        2. Set sync with for satellite ID to be active/collector
        3. Enter the Sync ID field for the remote site
    5. Print a custom report for all recs in the store table to get IDs
    6. Choose prefs > sync
      1. double click the site in question. Click generate sync records for site
        1. Choose the option- new store
        2. Records will be generated for items, names etc and will sync to the satellite
      2. run Footrunner code:
        1.  sync_create_records_for_store (<store_id>;"THIS_IS_THE_UUID_OF_THE_STORE_ON_PRIMARY")  \\ ALERT("Bhayo")

Previous: Reminders Next: Transaction Codes

  • other_stuff/remote_sync.txt
  • Last modified: 2018/11/23 03:41
  • by mark