Development

In this part topics related to the actual reference set development process are presented.

Prerequisites for Creating a Reference Set

As part of developing a new reference set it should be created and identified as part of the Edition that it belongs to.

When creating a new reference set, the organization developing the reference set will need access to a namespace in order to generate SCTIds. Within their namespace, a moduleId concept (with an FSN and Preferred Term) should be added, placed in the 900000000000443000 | Module (core metadata concept)| subhierarchy within the metadata, for each authoring organization.

To specify any reference set it is necessary to identify the reference set by creating a concept and name the reference set by associated descriptions. The concepts should be a subtype of the relevant reference set type concept in the foundation metadata hierarchy. This could for example be as a descendant of the 446609009 | Simple type reference set (foundation metadata concept)| , or the 900000000000512005 | Query specification type reference set (foundation metadata concept)| . The latter is used for intensional reference set definitions. Following table shows the steps to follow when creating a new reference set.

Authoring Reference Sets

Authoring reference sets in an extension may involve creating a new reference set, inactivating a reference set or modifying a reference set. Additionally, reference set members may also be added, modified or inactivated in an extension.

The sections that follow will examine the purpose, principles and process for each of these authoring tasks.

Authoring Reference Sets

• Create a New Reference Set

• Modify a Reference Set

• End Support for a Reference Set

Authoring Reference Set Members

• Add Members to a Reference Set

• Modify Members of a Reference Set

• Remove Members from a Reference Set

Create a New Reference Set

Principles

When creating a new reference set it is important that the reference set metadata concept (which identifies and names the reference set) and the associated reference set members belongs to a module of the reference set producer. I.e. if a reference set is created as part of an extension, the reference set metadata concept should belong to a module within the namespace of the extension producer.

All reference sets follow a logical model. Each reference set type is represented by a specific data structure which enables a specific functionality. SNOMED International specifies a set of reference set types, which can be used to support a set of terminology management purposes. SNOMED CT Members (National Release Centers) and Affiliates may also create reference sets to assist with localization and effective implementation of SNOMED CT. In general, an extension producer can create a reference set within their extension, but the reference set type should be defined in either

• • The producers extension

  • The International Edition

  • Other extensions (which the producers extension modules are dependent on)

For more information about the different reference set types and their data structures, please refer to the reference set release file specification.

Please note that for all reference sets, there should be guidance on what is possible and what is not possible. For some International reference sets, it is acceptable and even required to add extension rows to the International reference set. For example, the Module Dependency Refset, the MRCM Module Scope Refset, and Language Refset. This is not necessarily applicable to all reference sets. Over time, further guidance in this area will be developed.

Process

The table below provides a summary of the process to follow when creating a new reference set.

Define the reference set in the metadata hierarchy

The following steps should be taken for creating the reference set concept:

Define the reference set Attributes within the metadata hierarchy

Add new concepts for each of the reference set member attributes, if necessary. If the reference set attributes describing the pattern are adequate to describe the reference set's attributes, then these can be used instead. You may add new concepts for some of the attributes, and reuse existing concepts for other attributes, if you wish.

Following steps should be taken for each attribute that you wish to create:

Create the Descriptor for the reference set

If the reference set does not follow an existing reference set pattern, the additional attributes specific for this customized reference set should be included in the Descriptor reference set.

Add members to the reference set

Reference set members are added to the reference set, which includes specifying the attribute values for each reference set member, see Add Members to a Reference Set.

Modify a Reference Set

Principles

Principles for modifying reference sets include:

• Reference sets can be modified by changing the value of the active attribute or the value of the moduleId attribute, i.e. for activating or inactivating the reference set, or moving the reference set to another module (a module on which the current module depends).

  • The reference set metadata concept is always primitive and the concept can therefore not be changed from primitive to fully defined, or from fully defined to primitive, i.e. the value of the definitionStatusId attribute should be primitive for all versions of the reference set

• The reference set type should not be changed, i.e. the |is a| relationship, linking the reference set to the appropriate reference set type concept should not be changed

  • The only situation where the the reference set type may change is if the reference set was originally misplaced in the SNOMED CT hierarchy.

• If the reference set needs to be changed to a different type, (for example, from a simple reference set to an ordered component type reference set) then it would require the original reference set to be inactivated and a new reference set to be created in accordance with the Descriptor for that reference set type.

• Reference sets that belong to a module which is not managed by the extension producer should not be modified.

Process

Modifying a reference set created within the extension module

The process for modifying the reference set concept, is similar to the general process for modifying concepts in SNOMED CT. Therefore, please refer to Modify Concept in an Extension for further guidance.

End Support of a Reference Set

Principles

When the owner of a reference set wants to end support for the reference set, this should be indicated by making the changes described in the process section below.

In the situation where the reference set belongs to an extension, the owner of the extension may only make these changes to a reference set that is currently in an extension module for which it is responsible. The exception to that rule is that, in the case where a responsibility for maintenance of a reference set is transferred to another organization, the organization to which responsibility is transferred is required to take some of these steps.

Prior to ending support for a reference set, it is important that the reference set producer has an overview of the extent to which the reference set is used. If a producer and owner of a reference set continues to distribute an unsupported reference set with active members, there is an inherent risk that it will continue to be used. However, deprecation formally inactivates the references set members to mim this possibility.

Process

The table below provides details and considerations on the process of inactivating a reference set.

Distribution of an Inactivated Reference Set and its Members

If the extension producer wants to avoid users from needing to import a deprecated or transferred reference set in future releases, the inactivated reference set may be separated from the main extension release (e.g. it could be in a separate release package, or accessible via a separate service or from a static location). Changes in packaging must be formally notified to users of the extension in advance of the change.

Authoring Reference Set Members

Add Members to a Reference Set

Principles

Specifying members of a reference set can be done in different ways. It will depend on the requirements for the reference set what approach is feasible and possible. Please refer to the Development Approaches and Development Methods sections for detailed instructions on the approaches and methods for identifying the SNOMED CT components that will be referenced by the members of the reference set.

However, each reference set member will be represented in the reference set in accordance with the following principles and process.

Reference set members may be added to reference sets belonging to modules in

• The producers extension

• The International Edition

• Other extensions (which the producers extension modules are dependent on)

Reference set members created in the extension should be created within the module of the extension producer, so that it is possible to distinguish the reference set members created within the producers extension from reference set members created by other organisations (i.e. belonging to other modules).

Referenced Components

Even though the individual reference set members belong to the module of the extension producer, the actual components that are referenced by the reference set member, may belong to modules in

• The International Edition

• The extension in which the reference set and its members are produced

• Other extensions (which the producers extension modules are dependent on)

Process

The table below provides a summary of the process to follow when adding members to a reference set.

Modify Members of a Reference Set

Principles

Principles for modifying reference sets include:

• Reference sets can be modified by

  • Adding or inactivating reference set members. Please refer to the guidance on Add Members to a Reference Set or Remove Members from a Reference Set.

  • Modifying mutable attribute values of reference set members. To see what attributes are mutable for each reference set type, please refer to the specification of the specific reference set type in the reference set release files specification. If the reference set type is a locally defined reference set, please consider any mutability constraints on the individual attributes.

• Don't modify immutable attributes of a reference set.

  • In the case where the modification of an immutable attribute is required, this should be done by inactivating the reference set member and creating a new reference set member with the required, updated values.

• If circumstances require you to modify reference set members that belong to another module than the producers extension, following options exist:

  • Inactivating the specific reference set member in your own module and create a new reference set member with the updated value

    • The benefit of this approach is that you retain the definition and representation of the reference set member as it was intended by its original authors, and the new reference set member will be easily identified as a local reference set member, as the identifier of that reference set member is not available in the original reference set

  • Create a new version of the specific reference set member in your own module, and make the necessary modifications

Process

The table below provides a summary of the process to follow when modifying an existing member in a Reference Set.

Table: Modify reference set member

Remove Members from a Reference Set

Principles

Any inactivation of reference set members should be done in the module of the reference set producer.

Process

The table below provides a summary of the process to follow when removing members from a reference set.

Development Approaches

Developing reference sets can be done in different ways, depending on requirements, resources and skills of the organization who wish to develop a reference set. In some situation it may be the best approach to build the reference set without making use of or relying on any previous work for assistance. In other situations, existing reference sets may meet the requirements for another organization that wants to develop a reference set, and in these cases there are different approaches to utilize the existing work. In this section we introduce different general approaches to the creation of SNOMED CT reference sets.

Develop New Reference Set

One way to create a reference set is to specify the content from SNOMED CT to be included in the reference set without looking at any existing SNOMED CT reference sets. This approach may be chosen, if no reference set is available that meets the requirements of the developing organization. It may also be because the requirements for the new reference set is so clearly defined or limited in scope that it is easier to create the reference set from scratch.

This could for example be, if a group of orthopedic surgeons are developing a subset of SNOMED CT procedure concepts to be included in a certain pick list of their local electronic health record. The group already know what options should be available in the pick list, so they choose to create their own reference set and add the required component references to this reference set, see figure below.

Depending on the scope of content for the reference set there are different development methods that can be applied for selecting, or defining, reference set members.

Developing a new reference set also requires the developing organization to make changes to the reference set as necessary to meet evolving requirements.

Adopt an Existing Reference Set

As the experience of the SNOMED CT community grows, it becomes increasingly likely that there is some existing work that will provide a useful starting point, or that can be used to validate the work that you are doing. In this case, adopting an existing reference set may be a useful approach to take for acquiring a reference set.

Adopting an existing reference set is when the adopting organization/project use the existing reference set and future updates of that reference set as provided. This means, that the adopting organization commits to adopting any changes that are made to the source subset over time.

To be able to adopt and existing reference set it is a prerequisite that the existing reference set is part of a module that is included in the SNOMED CT Edition used by the adopting organization.

It is also a prerequisite that those maintaining the source subset publish changes in a predictable fashion, and the adopting project has processes in place to manage this adoption.

Adopting an existing reference set is an attractive solution in situations where the source subset is already being used within the communicating community, and so avoids the need for mappings to be maintained, or other strategies developed to deal with differences.

The drawback is that if the adopting project requires changes to the contents of the subset it does not have any mechanism for requesting such changes or making them happen.

Copy an Existing Reference Set

In the situation where an existing reference set meets the requirements of an organization who wish to use a SNOMED CT reference set, it may be a solution to copy this reference set instead of adopting the reference set. Copying a reference set is a useful approach in the situation where the existing reference set is part of a Module which is not included in the SNOMED CT Edition that the copying organization use.

Copying an existing reference set means create a new reference set with members referencing the same components as the existing reference set.

It is important to have a clear strategy for maintenance of a copied reference set. When an organization chooses to copy an existing reference set they need to make changes to the reference set as necessary to meet their evolving requirements. Whether it is the copying organization or the authoring organization who is responsible for adding or inactivating content to the existing reference set depends on the agreement between the involved parties. It also depends on whether the authors of the existing reference set have established a process, which deals with requests for changes. In either situation the copying organization will need to apply the changes made to any new version of the original reference set to the copied reference set.

Adapt an Existing Reference Set

Another approach to develop a reference set, is to adapt an existing reference set. This is where an existing reference set is used as a source of inspiration for the developing organization. It may also be, that the existing reference set almost meet the requirements of the adapting organization, but some modifications are needed. The adapting organization may wish to add more content to the existing reference set, or there may be content in the existing reference set which should be excluded from the new reference set.

Like copying reference sets, adapting a reference set will include the creation of a new reference set which is authored and maintained by the adapting organization. The members of the new reference set may then reference components that are also referenced in the existing reference set, they may exclude references that are in the existing reference set, or they may add other component references to meet the requirements of the adapting organization.

Development Methods

When developing simple reference sets, there are different methods which can be applied to determine what concepts to be included in the subset. The two overall approaches to defining the reference set members are manual enumeration and intensional definition.

Requirements for Manual Enumeration

Identifying reference set members by manual enumeration is useful when the number of members required is limited or when migrating from a well-specified domain. The source information may come from a variety of sources, such as:

• Subsets of concepts from other code systems

• Legacy codes (when migrating from a system that do not use SNOMED CT)

• Clinical guidelines and other knowledge resources

Manual enumeration is also useful when the reference set should be stable over time, i.e when addition of concepts, in subsequent releases of SNOMED CT, is not required to be reflected in the reference set.

Requirements for Intensional Definition

Identifying reference set members intensionally is useful for creating subset of concepts who share a set of common characteristics, such as:

• Being a descendant of the same focus concept

• Sharing one or more attribute relationships

Intensional definition of reference set members is also useful when changes to concepts, in subsequent releases of SNOMED CT, is required to be reflected in the reference set.

Review and Quality Assurance

Ensuring the quality of a reference set is important for the successful use of SNOMED CT. It is crucial that the content of the reference set references the concepts which represent the meaning of the data it is to be linked to. In addition, it is important that the human interpretation of the concepts referred to aligns with the logical definition provided by SNOMED CT. Even when using reference sets for less patient safety critical purposes, the quality of the reference set needs to be sufficient to be trusted to serve its purpose.

Stages of Quality Assurance

Reviewing a reference set is important throughout the reference set development process, however at least three types of validation should be emphasized, see illustration below.

• Design review: The overall objective with this review is to verify whether the reference set design meets the requirements.

• Content review: The overall objective with this review is to verify whether the selected reference set members are sufficient for the context where the reference set is to be used.

• Test: The overall objective with testing the reference set is to validate the reference set in the context where it is to be used. Testing is done to assure that the reference set meets the needs of the involved stakeholders.

Review

Reviewing a reference set before testing it in the context of its usage is important in order to correct the most distinctive errors, such as content gaps or disagreements about selection of concepts. Additionally, reviewing a reference set is important, because it is much easier to make major adjustments as early as possible. In addition, the adaptability of the reference set increases the higher the quality of the reference set, according to the perspective of the subset users. The level of quality assurance needed will of course be dependent on the practical use case of the reference set.

The checks that should be done as part of the review process include:

1. Check that all concept ids are current (that there is no references to inactive concepts)

2. Check that all concepts included in the reference set are appropriate to the use case

3. Check that the terms used to describe these concept ids are appropriate

4. Check that there are no gaps in content. I.e. concepts that should be included in the reference set that aren't

5. Check that there is clear context and meaning, which align with the surrounding models that the reference set is to be used with

Skills and Roles

The people involved with reviewing the reference set should be by a combination of domain experts and SNOMED CT experts.

• Domain experts: There may be different types of domain experts, but at least two types of domain experts can be specified :

  • Clinical domain experts: Persons who have knowledge about the use of the reference set and knowledge about the context where the subset is going to function. Often, clinical domain experts will be clinical users who have the sufficient level of clinical knowledge, to evaluate whether the scope and the content of the reference set is sufficient.

  • Technical domain experts: Persons who have knowledge about the technical environment where the reference set is going to be implemented. technical experts should have knowledge about the various models to which this reference set should be bound. Often, these experts will have a technical background, for example software architects, programmers, IT-managers. These experts are especially important at an early stage of the reference set development process, to review and validate the design of the reference set in relation to the requirements set out by the technical infrastructure.

• SNOMED CT Experts: Persons who have the sufficient knowledge about SNOMED CT to review the concepts selected for a specific purpose. These persons should be able to assess whether the definition of the selected reference set members suite the domain of the reference set. For example, if the reference set represents a problem list, then the included members should reference clinical finding concepts. This group is also serve an additional, supportive role in terms of guiding the domain experts in reaching correct interpretation of the reference set structure and content.

Review approaches

SNOMED International does not recommend ONE specific approach for review of reference sets, but blinded approaches are typically a feasible approach to achieve high quality reference set. Alternatively, or additionally, a combination of review methods can be recommendable to ensure a reliable and usable review process. Ideally, reference sets should be reviewed iteratively until the reviewers are satisfied. Examples of the approaches to take are illustrated in the following table.

Reference set review approaches

Test

In addition to the abstract assurance of the subset in isolation, there may be a requirement for a level of testing to be undertaken in healthcare systems, and tested under the exact circumstances of intended use. This may be undertaken by releasing a technology preview targeted at specific bodies for feedback. An impact assessment and, most importantly, safety testing will need to take place upon deployment of the subset into systems, particularly where significant changes have taken place.