--------------------------------------------------------------------------------
Steps to make a release:
--------------------------------------------------------------------------------

Prep for a new release; get and store cloud share links
--------------------------------------------------------------------------------
For EACH package doing a new release for (installer, program, reflib, tools):

(1) Check its respective <package>.json file for the correct NEW version and 
    date. Possibly bumping up the version # by one and setting a future date. 
    Modify the URL to the new expected name of the file based on the version 
    number and date.

(2) Touch this planned NEW release ZIP archive file name (new ver, new date) 
    (or copy an existing latest ZIP archive file and rename) in MS OneDrive.

(3) Get this newly created ZIP archive file share URL from MS Onedrive.

(4) In the https://get.wgse.io/.htaccess file, add the name of this new release 
    file and point it to the share URL. Note that the download URL is different 
    than what the share URL utility gives. Take just the unique key and paste it
    into a copy of an example download share URL already in .htaccess

(5) When doing this for the installer package, only do it for each track you are
    testing / making.

FOR THE INSTALLER PACKAGE ONLY (if it is to be updated)
(6) For testing, make your own local release-override.json file that points to a 
    local latest_release.json file.
    Copy your edited latest_release-<track>.json file to that test local 
    location. This makes the installer, when run, look in your local 
    latest-release JSON file for the new, test release package URLs.

    In make_release.sh, if "release" is selected, then the "latest: (generic 
    name) JSON and installer.zip files will be created and copied to the MS 
    Onedrive share; thus making the new release live for all users. When doing 
    that, make sure to first delete your release-override.json file so it is not
    included in the released installer package.

(7) Edit scripts/make_release.sh for any changes in the dev/ release folder 
    (things to remove; things to keep). If you discover something that the 
    installer needs to handle, edit the scripts/zinstall_common.sh and maybe
    scripts/zuninstall_common.sh to handle the change (e.g. remove any old files
    that will not get replaced, renamed, etc in the version updates)

(8) Do an incremental backup of the development directory for safety
 
Loop until satisfied:
--------------------------------------------------------------------------------
(1) Run scripts/make_release.sh <package> <track> (as desired) on a MacOS (only)

(2) It will create a local ZIP file and maybe latest-release JSON file.  One you
    can copy; possibly to the MS Onedrive server for testing. Thus overwriting 
    the "touched" files made earlier. You should replace the existing files if 
    everything is done correctly. Thus not changing the already captured share 
    URL's edited into the <package>.json file(s) that were stored inside the 
    newly created ZIP(s) and latest-release JSON.

(3) Copy your Installer to the desired location and install as normal.  The 
    release-override.json should find your local latest-release JSON file for 
    the definition of packages needed for this test release.

(4) Verify the new packages and URLs work, that the installers do a fresh 
    install, and installers do an update install (properly) on EACH PLATFORM 
    (OS, OS Version, CPU Arch). Also that the Uninstaller works for various 
    options and relocations of the reference library.

(5) Do any regression testing on WGS Extract and Library functionality itself.

note: You can help others test it by providing them the release-override.json 
file. And the latest-release file it points to if not already pointing to a 
cloud server file.

*** loop until satisfied. ***

Make a release:
--------------------------------------------------------------------------------
(1) Run scripts/make_release.sh again but this time with "release" instead of 
    "installer". Make sure to delete or edit the release-override.json file 
    first. This will copy the new latest release JSON and ZIP to the MS Onedrive
    server.

(2) Notify TRACK testers of a new release in the Facebook group. Only ever
    distribute the Bitly short links to the latest_installer.zip file that is 
    static and does not change.  Anyone simply runs their installer to get the
    new release.  New users get the new installer when they first download from
    the Bitly link.

TBD  Sha256 and md5 checksums on the main release page when installers updated?
  When other packages updated? Put them in the latest_release.json files with 
  the updated URL's?  Catch 22 as you have to make them to get the signature.
  Unlike the URL where you can use a dummy file copy to get the share URL.


--------------------------------------------------------------------------------
To check a development release:
--------------------------------------------------------------------------------
To check a development release BEFORE posting it as a new version, create a 
 release-override.json with a URL to a local location for the latest-release 
 JSON file. The installer will then take the new location to look for a 
 latest-release json file locally. And thus get the unreleased version numbers 
 and URLs to the packages to test. The local release.json file in the 
 development directory is never used to create an installer package. It is
 created from scratch by make_release.sh. And the installer will never overwrite
 an existing release.json file when installing. The release.json file stored in
 the installer is created from stored generic "latest" URLs in make_release.sh

You must make sure to remove the release-override.json file BEFORE making the 
final release for users.

Example entries you can use in the release-override.json file instead of the 
standard https://get.wgse.io/... URLs are:
    file:///wgse/dev/latest-release-Dev.json               (a local file)
    file:////randy-pc2/wgse/dev/latest-release-Dev.json    (a network file)
    file:///c:/wgse/dev/latest-release-Dev.json            (a local file with 
                                                     specific disk on Windows)

Generally, during development, the command to run this script is 
 "scripts/make_release.sh program" which will create the _program.zip package 
 archive. Likely you will also run "scripts/make_release.sh installer Dev" to 
 create a new _Dev_installer.zip archive (if there were updates to the 
 installer). You can run "scripts/make_release.sh allpacks alltracks"
 to make everything and then only upload what has changed. Key is, you need to 
 edit / confirm the content of each of these 4 package version json files for 
 the appropriate version number (bump by one?) and URL. And then the other 2 (4)
 for Windows Cygwin64 (Msys2) must be changed in their respective make_ scripts 
 before they are run.


--------------------------------------------------------------------------------
Background details on the new release and version control system in v5:
--------------------------------------------------------------------------------
(Make sure any links in the dev folder are hardlinks to directories. That is, junctions in windows terminology. NTFS
 volumes mounted to MacOS only follow junctions. MacOS bom creation will fail if symbolic. In windows cmd.exe:
 "mklink /j link_name target_path" to make a junction; "dir" to tell you which type of link is used; Windows explorer
 to safely remove a junction link without removing the target.  Under l/uinux, "find . -links +1" reports hardlinks
 whereas "ls -ls" will show softlinks.)

A version control system is in place.  The URL to a versioned ZIP archive is stored in a version file which are
 themselves stored in the ZIP files created.  So you have to fake-create the new ZIP file and get its share URL,
 edit the URL into the appropriate JSON file, and then make the actual release. Catch22. Note that the share URL
 is to the actual file and not file name. So you can rename the target file and move it to a different folder and
 the URL will not change. Currently, all share URLs are on the MS Onedrive archive as Google Drive makes it near
 impossible to programatically download files larger than 25 MB.

All the 8 JSON version files are collected together into a latest-release JSON for each release track. That file
 is only ever checked into the github repository location pointed to by a release.json file.  The release.json
 file is stored in each installer and set with the appropriate track for that installer.  That is the only
 difference between each installer file.  The release.json file in an installer will only be copied on new
 installations.  If it already exists, it will not be overriden. (See special note on how to make the installer
 force pick-up the new release.json in the ZIP archive.)

A top level file (release.json) in the installer package ZIP archive contains the "selected" release track of
 either Beta, Alpha or Dev(eloper).  It also stores the URL to find the latest-release JSON file for each track.
 The canonical release.json file is stored in the make_release.sh file here. As it creates each release.json from
 scratch when making each installer. The default URL to the latest-release JSON stored there is to the github
 check in of the latest-release.json file.

The make_release.sh file also has stored the default URL to be stored in each install.json file for each track.
 When making a a track installer, the current install.json file is left as is with its stored URL.  When making
 a release installer, the URL in the install.json file is overriden by that contained in the make_release.sh.
 Make_release.sh stores a URL that points to a "latest" installer for each track located on the MS Onedrive share.
 So when making an installer release, the install.json file stored in the installer ZIP archive will be updated
 with this new URL. So the "latest" installer file for the track on the MS Onedrive must be overwritten by the
 newly created installer.

Each of the six mutex ZIP archive packages are independently version controlled with their own JSON files included
 in their ZIP archive.  The package jsons have a version number, date created, and the URL to find the online ZIP
 archive file (on MS Onedrive). This allows us to compare a current, installed release (JSON file) to the latest
 available (online, latest-release JSON file) to decide whether the package needs to be updated. And if updating,
 the new URLs are found in the online, latest release JSON file.

There are six versioned packages (four for everyone; two for just windows):

  Version control files            Package      Suffix of ZIP   Explanation
  -------------------------------  -----------  --------------  -----------------------------------------------------
  scripts/installer.json           installer    _installer.zip  WGS Extract Installer / Uninstaller with scripts/
  program/program.json             program      _program.zip    WGS Extract main Program release (python, etc)
  reference/reflib.json            reflib       _reflib.zip     WGS Extract Reference Library subsystem
  jartools/tools.json              tools        _tools.zip      WGS Extract Local tools (jartools/, yleaf/, FastQC)

  cygwin64/usr/local/bioinfo.json  bioinfo      _bioinfo.zip    (Windows only) Cygwin64 port of Bioinformatic tools
  cygwin64/cygwin64.json           cygwin64     _installer.zip  (Windows only) Cygwin64 base release
  (no json; date matches others)   -            _builder.zip    (Windows only) Cygwin64 base release with compilation
                                                                    environment to make _bioinfo release. Package
                                                                    not distributed but URL to it in the installer.

  release.json                     installer    (in installer)  Stores latest-relese JSON URLs per track. Also has
                                                                    the selected track stored in it.
  release-override.json            installer    (in installer)  Override copy of release.json file that will replace
                                                                    any release.json file during install.

  latest-release-Dev.json          release      -               One for each track; generated in make_release and
                                                                uploaded when appropriate to get.wgse.bio. Contains
                                                                copies of all JSON files that make that track release.

  Release Make files               Package      Suffix of ZIP   Explanation
  -------------------------------  -----------  --------------  -----------------------------------------------------
  make_release.sh / .txt           program (but deleted)        These files. Stored in scripts/. Creates a
                                                                    release.json file from scratch for each installer.
                                                                    Copies release-override.json into installer if
                                                                    exists.

  cygwin64/make_cygwin64.sh         cygwin64                    Stores the URL and version for the new cygwin64
                                                                    release and creates the cygwin64.json file to
                                                                    include with a release.
  cygwin64/usr/local/make_bioinfo.sh bioinfo                    Stores the URL and version for the new bioinfo tools
                                                                    release on Windows; creates bioinfo.json file.

  msys2/make_msys2.sh               msys2                       Stores the URL and version for the new msys2
                                                                    release and creates the msys2.json file to
                                                                    include with a release.
  msys2/ucrt64/make_bioinfo.sh      bioinfo-msys2               Stores the URL and version for the new bioinfo tools
                                                                    release on Msys2 Windows; creates
                                                                    bioinfo-msys2.json file.

The first four JSONs are the source for that packages version, date and URL.  They are assumed current in the
 development directory and simply copied from there into the appropriate package archive. The archive .zip filename
 is created from the version and date stored in the JSON.

The last two cygwin packages are created by the make_cygwin64 and make_bioinfo.sh scripts. The scripts contain the
 version and URL and so must be edited to update the version. The generated zip file names start with cygwin64_.
 There is a third, undistributed _builder.zip generated that contains the additional cygwin64 files needed to build /
 compile the bioinformatics tools for the bioinfo package on Windows.  Ditto packages for Msys2 environment.

The installer consults the release.json file in the local installation to find the track type and the corresponding
 latest-release URL. It then pulls in the (online) latest-release json file with all 6 (8) packages' version info.
 This latest is used to compare to any currently installed package (determined by the local json files of the
 installed release as shown above). The above 6 (8) version jsons are concatenated into one file named
 latest-release-TRACK.json (created by make_release.sh and possibly uploaded to make the release available to users).
 If the file release-override.json exists, the installer ignores any local release.json and any in the installer
 package and instead copies the release-override.json file into its place.

The packages are mutually exclusive in their content.  The merger of the packages creates the older full releases
  used in v3 and earlier. The package content are:

(1) The Reference Library package is a large, binary blob and has historically been most of the release size. The
 Library does not update often.  So this allows for a separate versioning and quick, reliable upgrade of just the
 Program package independent of the more static Reference Library elements.  The microarray templates are now  in
 the Reference Library package.  Historically, a few key genomes  were  shipped  within  it  also.  Now  all  are
 dynamically downloaded when needed during program operation.

(2) yleaf and its templates are kept together and redistributed with jartools/Haplogrep  and  FastQC;  the  three
 as the Tools package.  The Tools package is now the largest after including the GATK and  DISCVRSeq  jar  tools.
 But, like the Reference Library, is not expected to update often.

(3) The Program package is mainly the program/ folder of python code as well as the user-initiated WGSExtract and
 Library scripts at the top level. It also includes the reference library manipulation scripts (not starting with
 z*sh) in the scripts/ folder. It is expected to change the most often but also has the smallest content like the
 Installer package.  It or key files in it will be auto-loaded whenever outdated in a future release.

(4) The Installer packages have the top-level Install / Uninstall scripts, top-level release.json, release notes,
 Readme,  amd open_source_licenses/ folder along with all scripts/z*.sh scripts.  There  is  a  unique  Installer
 package for each track (Beta, Alpha, and Dev-elopment) as the release.json and scripts/installer.json files have
 unique entries for that particular track. It may contain a release-override.json file during development or when
 we need to guarantee replacement of a local release.json file a user may have already stored.

get.wgse.io stores the 3 different latest-release JSON files for each track.  The default release.json file in each
 installer package points to this location.  wgse.bio has links to each of the release track "latest" installers
 on get.wgse.io. The URL in the installer.json file is never really used. It is always obtained from a latest-release
 json file only. The stored URLs to get.wgse.io are actually HTTP rewrites (redirections) on get.wgse.io to MS
 Onedrive links for the actual files -- JSONs and package .zips.

get.wgse.io was introduced in v4.42 and later. Previously, the latest-release JSON files were stored on the
 wgsextract-dev.github.io server. The latest-release installer .zip links on wgsextract.github.io were bitly URL
 shortened links to their google drive storage location. Before the release system introduced in v4.xx, the google
 drive URLs for packages were hardwired into the scripts.

We started with google drive cloud storage for distribution. But as they became more problematic for upload and
 download of large files (even on a paid subscriptions; both for browser and programmatic access), we switched to
 ms onedrive. Especially after bitly started accepting ms onedrive links for shortening. But introducing get.wgse.io
 got rid of the need for bitly link shortening.

