DICOM storage commitment ¶

Contents

DICOM storage commitment

Introduction ¶

Starting with release 1.6.0, Orthanc implements DICOM storage commitment, both as SCP and as SCU (i.e. both as a server and as a client).

Storage commitment is a feature of the DICOM standard by which an imaging modality “A” asks a remote imaging modality “B”, whether “B” accepts responsibility for having stored a set of DICOM instances.

Typically, a storage commitment request is issued by “A” after “A” has sent images to “B” using the DICOM C-STORE command. If “B” answers that all the images have been properly received, the modality “A” has the guarantee that the C-STORE commands ran fine, and thus “A” could decide to remove the images from its local database. If “B” answers that there was an error, “A” could decide to send the images again.

For more technical information, one may refer to the storage commitment Information Object Definition and Service Class in the DICOM standard. Orthanc follows the objective of the IHE Technical Framework regarding the Storage Commitment transaction (RAD-10). Following this IHE specification, Orthanc only implements the Storage Commitment Push Model SOP Class, both as an SCU (“Evidence Creator”) and as an SCP (“Image Manager”).

Orthanc makes the assumption that the storage commitment responses are sent asynchronously, which corresponds to most implementations of storage commitment.

Storage commitment SCP ¶

Overview ¶

Here is a diagram that outlines how storage commitment works in Orthanc:

In this sequence, three DICOM associations are used: The first one is the usual command to send the DICOM images from some SCU to the Orthanc SCP (C-STORE), the second association is the one by which the SCU asks the Orthanc SCP to process a storage commitment request (the SCU provides a list of DICOM instances to be checked by specifying their SOP instance UID and their SOP class UID), and the third one is the storage commitment response coming from the Orthanc SCP. The response is sent asynchronously from the Orthanc SCP to the SCU, once the storage commitment request has been processed by Orthanc.

The list of the DICOM modalities from which Orthanc accepts incoming storage commitment requests is specified in the configuration file of Orthanc, through the DicomModalities option. It is possible to disable storage commitment for selected modalities by setting their dedicated Boolean permission flag AllowStorageCommitment to false.

As can be seen in the figure above, the storage commitment SCP of Orthanc takes advantage of the jobs engine that is embedded within Orthanc. Whenever Orthanc receives a storage commitment request, it internally creates a job with a dedicated type (namely StorageCommitmentScp). This job can be controlled using the REST API of Orthanc, just like any other job. As a consequence, an external software is able to monitor, cancel or pause incoming storage commitment requests, by inspecting the list of jobs whose type is StorageCommitmentScp.

Sample usage ¶

In this section, we show how to query the storage commitment SCP of Orthanc from the command-line tool stgcmtscu. This free and open-source tool is part of the dcm4che project and emulates the behavior of a storage commitment SCU.

Firstly, we define one DICOM modality corresponding to stgcmtscu by creating the following configuration file for Orthanc:

{
  "DicomPort" : 4242,
  "DicomModalities" : {
    "scu" : [ "STGCMTSCU", "127.0.0.1", 11114 ]
  }
}

Secondly, we start Orthanc using the just-created configuration file:

$ ./Orthanc --verbose storage-commitment.json

We’ll be using some sample file /tmp/DummyCT.dcm, whose DICOM tags “SOP instance UID” and “SOP class UID” can be retrieved as follows:

$ dcm2xml /tmp/DummyCT.dcm | grep -E '"SOPInstanceUID"|"SOPClassUID"'
<element tag="0008,0016" vr="UI" vm="1" len="26" name="SOPClassUID">1.2.840.10008.5.1.4.1.1.4</element>
<element tag="0008,0018" vr="UI" vm="1" len="54" name="SOPInstanceUID">1.2.840.113619.2.176.2025.1499492.7040.1171286242.109</element>

Thirdly, we use stgcmtscu to get the status of one sample DICOM file. Here is what can be read at the end of the logs of stgcmtscu:

$ /home/jodogne/Downloads/dcm4che-5.20.0/bin/stgcmtscu -b STGCMTSCU:11114 -c ORTHANC@localhost:4242 /tmp/DummyCT.dcm
[...]
18:14:22,949 DEBUG - STGCMTSCU<-ORTHANC(2) >> 1:N-EVENT-REPORT-RQ Dataset receiving...
18:14:22,949 DEBUG - Dataset:
(0008,1195) UI [2.25.250402771220435242864082979068071491247] TransactionUID
(0008,1198) SQ [1 Items] FailedSOPSequence
>Item #1
>(0008,1150) UI [1.2.840.10008.5.1.4.1.1.4] ReferencedSOPClassUID
>(0008,1155) UI [1.2.840.113619.2.176.2025.1499492.7040.1171286242.109] ReferencedSOPInstanceUID
>(0008,1197) US [274] FailureReason
(0008,1199) SQ [] ReferencedSOPSequence

As can be seen, the SOP class/instance UIDs of /tmp/DummyCT.dcm are reported by the Orthanc SCP in the FailedSOPSequence field, which indicates the fact that Orthanc has not stored this instance yet. The FailureReason 274 corresponds to status 0x0112, namely “No such object instance”.

Fourthly, let’s upload the sample file to Orthanc, then execute stgcmtscu for a second time:

$ storescu localhost 4242 /tmp/DummyCT.dcm
$ /home/jodogne/Downloads/dcm4che-5.20.0/bin/stgcmtscu -b STGCMTSCU:11114 -c ORTHANC@localhost:4242 /tmp/DummyCT.dcm
[...]
18:19:48,090 DEBUG - STGCMTSCU<-ORTHANC(2) >> 1:N-EVENT-REPORT-RQ Dataset receiving...
18:19:48,090 DEBUG - Dataset:
(0008,1195) UI [2.25.141864351815234988385597655400095444069] TransactionUID
(0008,1199) SQ [1 Items] ReferencedSOPSequence
>Item #1
>(0008,1150) UI [1.2.840.10008.5.1.4.1.1.4] ReferencedSOPClassUID
>(0008,1155) UI [1.2.840.113619.2.176.2025.1499492.7040.1171286242.109] ReferencedSOPInstanceUID

The instance of interest is now reported in the ReferencedSOPSequence tag, instead of FailedSOPSequence. This shows that Orthanc has properly received the sample instance.

Plugins ¶

The Orthanc core implements a basic storage commitment SCP. This basic handler simply checks for the presence of the requested DICOM instances in the Orthanc database, and makes sure that their SOP class UIDs do match those provided by the remote storage commitment SCU.

For more advanced scenarios, it is possible to override this default SCP to customize the way incoming storage commitment requests are processed by Orthanc. This customization is done by creating an Orthanc plugin.

The custom storage commitment SCP is installed in the Orthanc core by using the OrthancPluginRegisterStorageCommitmentScpCallback() function of the plugin SDK.

Importantly, this primitive frees the plugin developer from manually creating the Orthanc jobs. One job is transparently created by the Orthanc core for each incoming storage commitment request, allowing the plugin developer to focus only on the processing of the queried instances.

Note that a sample plugin is also available in the source distribution of Orthanc.

Storage commitment SCU ¶

As written above, Orthanc can act as a storage commitment SCP (server). It can also act as a storage commitment SCU (client), which is discussed in this section. Here is the corresponding workflow:

Note that depending on the type of long-term archive media (hard disk, optical disk, tape, hard drive, cloud provider…), the storage commitment report (DICOM command N-EVENT-REPORT) may be sent long time after Orthanc has sent its storage commitment request (DICOM command N-ACTION), which necessitates Orthanc to handle reports asynchronously.

The active storage commitment reports are stored in RAM only, and are lost if Orthanc is restarted. The configuration option StorageCommitmentReportsSize sets the limit on the number of active storage commitment reports in order to avoid infinite memory growth because of the asynchronous notifications (the default limit is 100): The least recently used transactions are removed first.

REST API ¶

Overview¶

As can be seen in the figure above, storage commitment SCU is governed by 3 new routes that were introduced in the REST API of Orthanc 1.6.0:

POST-ing to /modalities/{scp}/storage-commitment initiates storage commitment requests. In this route, {scp} corresponds to the symbolic name of a remote DICOM modality, as declared in the DicomModalities configuration option of Orthanc.
GET-ing on /storage-commitment/{transaction} retrieves the status of a previous storage commitment request. In this route, {transaction} corresponds to an identifier that is available in the output of the call to /modalities/{scp}/storage-commitment.
POST-ing on /storage-commitment/{transaction}/remove asks Orthanc to remove the instances that have been reported as successfully stored by the remote SCP. This route is only available for fully successful storage commitment reports.

In addition, the route /modalities/{scp}/store that is used to send one file from Orthanc to another modality, accepts a new Boolean field StorageCommitment. If this field is set to true, a storage commitment SCU request is automatically issued by Orthanc after the C-STORE operation succeeds.

Triggering storage commitment SCU¶

We’ll be using a sample configuration file that is almost the same as for the SCP samples, but in which we declare a remote SCP instead of a remote SCU (only the AET changes):

{
  "DicomPort" : 4242,
  "DicomModalities" : {
    "scp" : [ "DCMQRSCP", "127.0.0.1", 11114 ]
  }
}

Given that configuration, here is how to trigger a storage commitment SCU request against the remote SCP, asking whether a single DICOM instance is properly stored remotely:

$ curl http://localhost:8042/modalities/scp/storage-commitment -X POST -d '{"DicomInstances": [ { "SOPClassUID" : "1.2.840.10008.5.1.4.1.1.4", "SOPInstanceUID" : "1.2.840.113619.2.176.2025.1499492.7040.1171286242.109" } ]}'
{
  "ID" : "2.25.77313390743082158294121927935820988919",
  "Path" : "/storage-commitment/2.25.77313390743082158294121927935820988919"
}

The REST call returns with the identifier of a storage commitment transaction that can successively be monitored by the external application (see below). A shorthand notation exists as well, where the JSON object containing the fields SOPClassUID and SOPInstanceUID object is replaced by a JSON array containing these two elements:

$ curl http://localhost:8042/modalities/scp/storage-commitment -X POST -d '{"DicomInstances": [ [ "1.2.840.10008.5.1.4.1.1.4", "1.2.840.113619.2.176.2025.1499492.7040.1171286242.109" ] ]}'

It is also possible to query the state of all the instances from DICOM resources that are locally stored by the Orthanc server (these resources can be patients, studies, series or instances). In such a situation, one has to use the Resources field and provide a list of Orthanc identifiers:

$ curl http://localhost:8042/modalities/scp/storage-commitment -X POST -d '{"Resources": [ "b9c08539-26f93bde-c81ab0d7-bffaf2cb-a4d0bdd0" ]}'

Evidently, the call above accept a list of DICOM instances, not just a single one (hence the enclosing JSON array).

Chaining C-STORE with storage commitment¶

Often, C-STORE SCU and storage commitment SCU requests are chained: The images are sent, then storage commitment is used to check whether all the images have all properly been received. This chaining can be automatically done by setting the StorageCommitment field in the corresponding call to the REST API:

$ curl http://localhost:8042/modalities/scp/store -X POST -d '{"StorageCommitment":true, "Resources": [ "b9c08539-26f93bde-c81ab0d7-bffaf2cb-a4d0bdd0" ]}'
{
   "Description" : "REST API",
   "FailedInstancesCount" : 0,
   "InstancesCount" : 1,
   "LocalAet" : "ORTHANC",
   "ParentResources" : [ "b9c08539-26f93bde-c81ab0d7-bffaf2cb-a4d0bdd0" ],
   "RemoteAet" : "ORTHANC",
   "StorageCommitmentTransactionUID" : "2.25.300965561481187126241492642575174449473"
}

Note that the identifier of the storage commitment transaction is part of the answer. It can be used to inspect the storage commitment report (see below).

Inspecting the report¶

Given the ID of one storage commitment transaction, one can monitor the status of the report:

$ curl http://localhost:8042/storage-commitment/2.25.77313390743082158294121927935820988919
{
   "RemoteAET" : "ORTHANC",
   "Status" : "Pending"
}

The Status field can have three different values:

Pending indicates that Orthanc is still waiting for the response from the remote storage commitment SCP.
Success indicates that the remote SCP commits to having properly stored all the requested instances.
Failure indicates that the remote SCP has not properly stored at least one of the requested instances.

After waiting for some time, the report becomes available:

$ curl http://localhost:8042/storage-commitment/2.25.77313390743082158294121927935820988919
{
   "Failures" : [
      {
         "Description" : "One or more of the elements in the Referenced SOP Instance Sequence was not available",
         "FailureReason" : 274,
         "SOPClassUID" : "1.2.840.10008.5.1.4.1.1.4",
         "SOPInstanceUID" : "1.2.840.113619.2.176.2025.1499492.7040.1171286242.109"
      }
   ],
   "RemoteAET" : "ORTHANC",
   "Status" : "Failure",
   "Success" : []
}

This call shows that the remote SCP had not received the requested instance. Here the result of another storage commitment request after having sent the same instance of interest:

$ curl http://localhost:8042/storage-commitment/2.25.332757466317867686107317231615319266620
{
   "Failures" : [],
   "RemoteAET" : "ORTHANC",
   "Status" : "Success",
   "Success" : [
      {
         "SOPClassUID" : "1.2.840.10008.5.1.4.1.1.4",
         "SOPInstanceUID" : "1.2.840.113619.2.176.2025.1499492.7040.1171286242.109"
      }
   ]
}

Removing the instances¶

If the Status field of the report equals Success, it is then possible to remove the instances from the Orthanc database through a single call to the REST API:

$ curl http://localhost:8042/storage-commitment/2.25.332757466317867686107317231615319266620/remove -X POST -d ''
{}

Plugins ¶

Thanks to the fact that Orthanc plugins have full access to the REST API of Orthanc, plugins can easily trigger storage commitment SCU requests as if they were external applications, by calling the functions OrthancPluginRestApiPost() and OrthancPluginRestApiGet().

Testing against dcm4che ¶

As explained in a earlier section about SCP, the dcm4che project proposes command-line tools to emulate the behavior of a storage commitment SCU and SCP. The emulation tool for the SCP part is called dcmqrscp. This section gives instructions to test Orthanc against dcmqrscp.

Let’s start Orthanc with an empty database and the configuration file we used when describing the REST API for the SCU:

$ ./Orthanc --verbose storage-commitment.json

In another terminal, let’s upload a sample image to Orthanc, generate a DICOMDIR from it (as the tool dcmqrscp works with a DICOMDIR media), and start dcmqrscp:

$ storescu localhost 4242 /tmp/DummyCT.dcm
$ curl http://localhost:8042/studies/b9c08539-26f93bde-c81ab0d7-bffaf2cb-a4d0bdd0/media > /tmp/DummyCT.zip
$ mkdir /tmp/dcmqrscp
$ cd /tmp/dcmqrscp
$ unzip /tmp/DummyCT.zip
$ /home/jodogne/Downloads/dcm4che-5.20.0/bin/dcmqrscp -b DCMQRSCP:11114 --dicomdir /tmp/dcmqrscp/DICOMDIR
15:20:09,476 INFO  - Start TCP Listener on 0.0.0.0/0.0.0.0:11114

In a third terminal, we ask Orthanc to send a storage commitment request to dcmqrscp about the study (that is now both stored by Orthanc and by dcmqrscp):

$ curl http://localhost:8042/modalities/scp/storage-commitment -X POST -d '{"Resources":["b9c08539-26f93bde-c81ab0d7-bffaf2cb-a4d0bdd0"]}'
{
   "ID" : "2.25.335431924334468284852143921743736408673",
   "Path" : "/storage-commitment/2.25.335431924334468284852143921743736408673"
}
$ curl http://localhost:8042/storage-commitment/2.25.335431924334468284852143921743736408673
{
   "Failures" : [],
   "RemoteAET" : "DCMQRSCP",
   "Status" : "Success",
   "Success" : [
      {
         "SOPClassUID" : "1.2.840.10008.5.1.4.1.1.4",
         "SOPInstanceUID" : "1.2.840.113619.2.176.2025.1499492.7040.1171286242.109"
      }
   ]
}

As can be seen, dcmqrscp reports that it knows about the study. Let us now create another instance in the same study by running a modification through the REST API of Orthanc, then check that dcmqrscp doesn’t know about this modified study:

$ curl http://localhost:8042/studies/b9c08539-26f93bde-c81ab0d7-bffaf2cb-a4d0bdd0/modify -X POST -d '{"Replace":{"StudyDescription":"TEST"}}'
{
   "ID" : "0b57dfc8-edb5f4c7-78f8bcdc-546908dc-b79b06f4",
   "Path" : "/studies/0b57dfc8-edb5f4c7-78f8bcdc-546908dc-b79b06f4",
   "PatientID" : "6816cb19-844d5aee-85245eba-28e841e6-2414fae2",
   "Type" : "Study"
}
$ curl http://localhost:8042/modalities/scp/storage-commitment -X POST -d '{"Resources":["0b57dfc8-edb5f4c7-78f8bcdc-546908dc-b79b06f4"]}'
{
   "ID" : "2.25.12626723691916447325628593043115134307",
   "Path" : "/storage-commitment/2.25.12626723691916447325628593043115134307"
}
$ curl http://localhost:8042/storage-commitment/2.25.12626723691916447325628593043115134307
{
   "Failures" : [
      {
         "Description" : "One or more of the elements in the Referenced SOP Instance Sequence was not available",
         "FailureReason" : 274,
         "SOPClassUID" : "1.2.840.10008.5.1.4.1.1.4",
         "SOPInstanceUID" : "1.2.276.0.7230010.3.1.4.8323329.16403.1583936918.190455"
      }
   ],
   "RemoteAET" : "DCMQRSCP",
   "Status" : "Failure",
   "Success" : []
}