Overview of blue/green deployment

Blue/green deployment is a method of deploying changes in a clustered server environment, while reducing production downtime and risk. This allows you to perform regular updates without stopping the entire production installation. This method gradually transfers user traffic from a prior version of an application or microservice to a new version, and thereby minimizes production downtime. In a blue/green deployment, you can stop an individual application server and deploy changes to that server without stopping the other servers in the cluster. The application continues to function during the deployment process.

Very large changes, or changes that are not supported by blue/green deployment, must still be deployed using the full deployment process, with the associated full deployment downtime and safeguards. You can perform a blue/green deployment of only one application at a time.

To allow changes with a blue/green deployment, the target (new) application build must be compatible with the source (old) application build. Use the Verify Blue/Green Compatible task in TeamCity to determine if the two application builds are compatible. If the builds are not compatible, then you must perform a full deployment. See Blue/green deployment compatible changes.

Blue/green deployment types

You can perform a blue/green deployment in the following ways:

• Single-click deployment

  A single-click blue/green deployment automatically advances through the deployment steps as long as verification checks are successful and no problems are encountered.

• Interactive deployment

  An interactive blue/green deployment pauses the deployment process at certain points and requires you to take explicit action to advance to the next step. This gives you the opportunity to review deployment reports and verify that there are no compatibility issues.

When you initiate the deployment, you will specify the deployment method.

Guidewire recommends that you perform an interactive blue/green deployment the first time you redeploy an application. With an interactive deployment, you can make sure it proceeds as expected and identify any issues. After you are comfortable with the deployment, you can switch to using single-click deployments to streamline the process.

Blue/green process

Before the deployment, a blue server cluster is running an InsuranceSuite application with your existing (base) configuration, and the green server cluster is offline.

After the deployment, the green server cluster is running an InsuranceSuite application with your new (target) configuration, and the blue server cluster is offline.

The following is a description of the primary activity in the InsuranceSuite blue/green deployment process:

Stage 1 – Before blue/green deployment

The blue server cluster is live and running your existing application configuration, and serving all production traffic.

Stage 2 – Initiating blue/green deployment

Initiating the deployment generates a database schema upgrade report, and then updates the database schema.

1. A green server is started to generate an upgrade report based on the new application configuration. The report details the file and database schema changes that will be made as part of the new deployment. The report also indicates if any schema changes are not compatible, and therefore whether the deployment can proceed or must be aborted. If performing an interactive blue/green deployment, you must review the schema migration report and approve the schema migration for the deployment to proceed. With a single-click deployment, as long as the migration report doesn't detect any problems, the deployment proceeds automatically without manual approval.

2. After verifying that the application target configuration is blue/green deployment compatible, one of the green server nodes is started to upgrade the database schema, and then is shut down again after the database schema upgrade is complete. No other changes are made to the database. After the database schema upgrade is complete, the blue server cluster continues to serve all production traffic and still has write access to the database.

3. The green cluster is enabled. If performing an interactive blue/green deployment, you must provide approval for this step to proceed.

4. The blue batch servers are shut down. Background tasks and batch processes are inactive.

5. The shutdown timer begins counting down. Users of the blue servers see a message telling them that the server will soon be restarted, and that they must finish their work and log out. During this time, they can continue working normally with the application user interface.

6. The green servers begin to start up.

Stage 3 – Transition from blue to green

1. Once the shutdown timer has expired and all green servers are ready, subsequent user requests are directed to the green servers.
2. The green servers accept new connections in read-only mode. Users can log in and view data, but can't make changes until the deployment is completed.

Stage 4 – Finalize blue/green deployment

The blue servers finish serving any in-flight requests and then begin to shut down. The green server cluster serves all the traffic.

The green server cluster is given write access to the database, and the router now directs all production traffic to the green server cluster.

Once all of the blue servers have shut down, the blue/green deployment can be completed. If performing an interactive blue/green deployment, you must provide approval for this step to proceed.

Blue/green deployment compatible changes

Guidewire supports most types of configuration changes during a blue/green deployment. Changes to SOAP and REST APIs and Gosu servlets are also generally compatible. All changes are blue/green compatible, other than those marked NON_COMPATIBLE by the Verify Blue/Green Compatible report. Non-compatible changes require a full deployment.

The following table shows examples of changes that are compatible and those that are not.

Note:

The examples below do not provide a complete description of compatible and non-compatible changes. For definitive and complete information about blue/green deployment compatibility, run the Verify Blue/Green Compatible report.

File or file path	Deployment compatibility
configuration > config > config.​xml	All changes are blue/green deployment compatible, except permanent and semi-permanent parameters.
configuration > config > extensions > entity	New entities are allowed. Adding nullable columns to existing entities is allowed. Some changes to element attributes in entity definition files (*.​eti, *.​eix, *.​etx) are allowed.
configuration > config > Integration	Adding, deleting, or modifying entries in the listed file types in the following directories and their sub-directories are blue/green deployment compatible: Integration > apis *.​swagger.​yaml published-apis.​yaml versions.​json Integration > filters *.​gql Integration > mappings *.​mapping.​json Integration > schemas *.​schema.​json versions.​json Integration > updaters *.​updater.​json Note: Guidewire generates the versions.​json files if you run gwb updateReleasedSchemaVersion at a command prompt.
configuration > config > Localizations	Adding, deleting, or modifying any of the display keys in the various display_languageCode.​properties files is blue/green deployment compatible. Changes to either language.​xml or localization_localeCode.​xml are not blue/green compatible, as these files must remain consistent across all cluster members.
configuration > config > Page Configuration	Adding, modifying or a deleting a PCF file are blue/green compatible.
configuration > config > Plugins	Plugin changes are blue/green deployment compatible.
configuration > config > resources > productmodel	Changes to product model pattern types are blue/green compatible.
configuration > config > Rule Sets	All changes to Gosu Rules, including additions, deletions, modifications, and renaming are blue/green deployment compatible.
configuration > config > servlet	All changes to Gosu Rules, including additions, deletions, modifications, and renaming are blue/green deployment compatible. IMPORTANT: Guidewire recommends that you undertake thorough testing after making a change to a Gosu servlet implementation to verify that all product integrations continue to work as intended.
configuration > config > typelists	Depending on the type of typelist, it is generally safe to add typecodes to an existing typelist or to add an entirely new typelist. It is also safe to edit the typelist description or change its category. However, it is not safe to make changes to LOB typelists outside of a very narrow context. Removing typecodes is not compatible with blue/green deployment.
configuration > gsrc	It is generally safe to change Gosu code during a blue/green deployment.

Risks associated with blue/green deployment

In performing a blue/green deployment, you are trading safety for convenience. In particular, there are risks associated with modifying any configuration related to distributed processing such as batch processes and work queues. Do not attempt to perform a blue/green deployment of the production application cluster if your changes affect those areas of the application.

It is possible for blue/green deployment to fail for a variety of reasons. If that happens, you must spend time and effort recovering. In the case of a full deployment failure, you would restore your backed up database and return to the pre-upgrade state. Because the system is down during the deployment, there is no risk of lost end-user work in the case of deployment failure.

To mitigate risk, it is important that your blue/green deployment be thoroughly tested in a pre-production environment.