Transfers accelerator plugin¶

Description¶

This plugin can be used to send local images to remote Orthanc peers, or to locally retrieve images stored on remote Orthanc peers.

The plugin also implements storage commitment, i.e. the peer that initiates the transfer is informed whether all the DICOM instances have been properly received by the other peer. The DICOM instances are individually validated using their MD5 checksum. In other words, this plugin provides atomicity in the transfers (i.e. a study/series is considered as a whole, and partial transfers are prevented).

Note that the protocol is entirely built over HTTP/HTTPS (and not directly over TCP), making it friendly with network firewalls and Web caches. Also, the plugin takes advantage of the jobs engine of Orthanc, so that transfers can be easily paused/canceled/resubmitted.

Technically, this plugin extends the REST API of Orthanc with endpoints that optimize the use of the network bandwidth over the HTTP and HTTPS protocols, through the combination of the following mechanisms:

• Small DICOM instances are grouped together to form so-called “buckets” of some megabytes in order to reduce the number of HTTP handshakes.
• Large DICOM instances are split as a set of smaller buckets in order to bypass nasty effects of TCP congestion control on low-quality network links.
• Buckets are downloaded/uploaded concurrently by several threads.
• Buckets can be individually compressed using the gzip algorithm, hereby reducing the network usage. On a typical medical image, this can divide the volume of the transmission by a factor 2 to 3, at the price of a larger CPU usage.
• Sending images to remote Orthanc peers can either be done with HTTP PUT requests (so-called “push mode”), or with HTTP GET requests if the local Orthanc server has a public IP address (so-called “pull mode”).

Compilation¶

$ mkdir BuildTransfers
$ cd BuildTransfers
$ cmake .. -DSTATIC_BUILD=ON -DCMAKE_BUILD_TYPE=Release
$ make

$ mkdir BuildTransfers
$ cd BuildTransfers
$ cmake .. -DCMAKE_BUILD_TYPE=Release \
           -DALLOW_DOWNLOADS=ON \
           -DUSE_SYSTEM_GOOGLE_TEST=OFF \
           -DUSE_SYSTEM_ORTHANC_SDK=OFF
$ make

Basic usage: Sending images¶

You of course first have to install Orthanc, with a version above 1.4.2. Secondly, you have to load the plugin and to declare the remote Orthanc peers in the configuration file. Here is a minimal example (obviously, adapt the parameters):

{
  "Name" : "MyOrthanc",
  "Plugins" : [
    "/home/user/orthanc-transfers/BuildTransfers/libOrthancTransfers.so"
  ],
  "OrthancPeers" : {
    "remote" : [ "http://1.2.3.4:8042/" ]
  }
}

Once Orthanc is running, when you open a patient, a study, or a series in Orthanc Explorer, you will see a new yellow button entitled Transfers accelerator. By clicking on this button, you will be able to send the local patient/study/series to one of the remote Orthanc peers (provided they are also equipped with the transfers accelerator plugin).

REST API¶

Here is a sample command line to receive a patient from the remote peer called remote:

$ curl -v -X POST http://localhost:8042/transfers/pull \
   --data '{
              "Resources" : [{"Level":"Patient","ID":"16738bc3-e47ed42a-43ce044c-a3414a45-cb069bd0"}],
              "Compression" : "gzip",
              "Peer" : "remote"
            }'

Note that several resources from different levels (patient, study, series or instances) can be retrieved at once.

Conversely, here is a sample command line to send the same patient to the remote peer remote:

$ curl -v -X POST http://localhost:8042/transfers/send \
   --data '{
            "Resources" : [{"Level":"Patient","ID":"16738bc3-e47ed42a-43ce044c-a3414a45-cb069bd0"}],
            "Compression" : "gzip",
            "Peer" : "remote"
          }'

The command above is the one that is issued by Orthanc Explorer under the hood (see section above).

Sending in pull vs. push mode¶

In the case DICOM instances are being sent to a remote peer, the plugin can work in two different modes:

• In “pull” mode, the plugin will transfer images by using as many HTTP GET requests as possible.
• In “push” mode, it will use a sequence of HTTP PUT requests.

Push mode is easier to setup, but pull mode should be favored, as it might lead to improved performance in the presence of Web caches. For pull mode to work, the remote peer must be able to make calls to the REST API of the local peer. This often means that the local peer has a public IP address.

In order to enable pull mode to send image from Orthanc peer “A” to another Orthanc peer “B”, 2 actions must be taken:

1. “B” must have “A” defined as one of its peers, by adding “A” to its OrthancPeers configuration section.
2. “A” must also have “B” defined as one of its peers, and the RemoteSelf property must be provided for peer “B”. This option specifies the symbolic name under which “B” is known to “A”.

Here is a sample configuration for “A”:

{
  "Name" : "A",
  "Plugins" : [
    "/home/user/orthanc-transfers/BuildTransfers/libOrthancTransfers.so"
  ],
  "OrthancPeers" : {
    "B" : {
      "Url" : "http://b.myinstitution.com:8042/",
      "RemoteSelf" : "A"
    }
  }
}

And here is a sample configuration for “B”:

{
  "Name" : "B",
  "Plugins" : [
    "/home/user/orthanc-transfers/BuildTransfers/libOrthancTransfers.so"
  ],
  "OrthancPeers" : {
    "A" : {
      "Url" : "http://a.myinstitution.com:8042/"
    }
  }
}

NB: Receiving images is always done in pull mode.

Advanced options¶

Besides the OrthancPeers configuration option, several advanced options are available to fine-tune the configuration of the plugin. They are listed below:

{
  ...
  "HttpTimeout" : 120,         // Can be increased on slow networks
  "Transfers" : {
    "Threads" : 6,             // Number of worker threads for one transfer
    "BucketSize" : 4096,       // Optimal size for a bucket (in KB)
    "CacheSize" : 128,         // Size of the memory cache to process DICOM files (in MB)
    "MaxPushTransactions" : 4, // Maximum number of simultaneous receptions in push mode
    "MaxHttpRetries" : 0,      // Maximum number of HTTP retries for one bucket
    "PeerConnectivityTimeout": 2 // HTTP Timeout (in seconds) used when checking if a remote peer has the transfer plugin enabled in /transfers/peers GET route
  }
}

Working with load-balancers¶

If the receiving Orthanc instance is implemented by a cluster of Orthanc instances behind a load-balancer, it is very important that all requests relating to a single “push” transfer target the same Orthanc instance.

In order to achieve this, in your load-balancer, you may use the sender-transfer-id HTTP header to route the requests. This header is populated in every outgoing HTTP request. By default, its value is a random uuid. If required, you may force the value of this HTTP header by adding a SenderTransferID field in the payload when creating the transfer:

$ curl -v -X POST http://localhost:8042/transfers/send \
   --data '{
            "Resources" : [{"Level":"Patient","ID":"16738bc3-e47ed42a-43ce044c-a3414a45-cb069bd0"}],
            "Compression" : "gzip",
            "Peer" : "remote",
            "SenderTransferID" : "my-transfer-id"
          }'