Approaches to extending the HSDS data structure

At the initial working group meeting for the next version of the Open Referral data specification (HSDS) there was some discussion of approaches to extending HSDS for different use cases. We don’t want this to delay getting an upgrade of HSDS out, but ultimately we need to understand and agree on an approach.

Two approaches are:

  • Filter a wider (extended) core data structure in different ways for different applications
  • Extended a core HSDS in different ways for different applications

In the UK we took the second approach when developing Open Referral UK. HSDS 2 adopted some of our extensions but in a slightly different way, so UK and US/international structures have incompatibilities.

Work so far on aligning (see US and UK Alignment and version control from February 2022) takes the second approach of having an “extended” HSDS with filters for different application profiles.

I’m keen on the “filter an extended HSDS” approach because it allows for greater re-use of tools developed and shared, such as the Open Referral UK validator. It avoids the same properties being describned differently by different publishers.

However, it does not have to be an either/or approach. Individual publishers can add their own properties which the validator ignores as long as these properties don’t conflict with the core specification. Open API feeds can be examined so we can assess if properties added would be useful to add to the HSDS standard in future versions.

The HSDS “servce attributes” and “other attributes” also allow for extension by referencing taxonomy terms that describe attributes. For future versions, we’ve suggested that we consider adding an optional “value” field to attributes. Again attributes added can be reviewed to see if anything is worth adding as a property in its own right in future versions of HSDS.

Thanks for this, Mike.

I think what makes this hard is that HSDS is used for interoperability inside contexts where certain aspects have to work in a particular way, and that particular way might be different between contexts: it would be great to have the UK validator be re-usable by other OR contexts, but we should also recognise that some features (I can imagine UPRN checks or AIRS taxonomy term lookups, for example) will always be context-specific.

I think that we broadly align on the question of how far we expect interoperability: the conversation so far has been much more about tools than data, so I think that we can focus on that.

Unsurprisingly, I guess it then comes down to a question of governance: how will we decide what gets in? How will we decide which of several, potentially incompatible, approaches to modelling certain concepts we use?

I don’t think that either a filter or an extension approach alone solves this; I think we just have to work through the consequences of our choice.

the validator ignores as long as these properties don’t conflict with the core specification.

This is good behaviour to have, IMO, but it is different from the behaviour that someone would experience taking an extended HSDS datapackage (which is how we currently ship HSDS) and running it through the OKFN datapackage validator. In practice, I don’t think anyone does this, or wants this “closed” behaviour - but we should be mindful of making sure that our tools match up with our expectations of the standard.

Thanks Rob. I think our views are broadly aligned.

some features (I can imagine UPRN checks or AIRS taxonomy term lookups, for example) will always be context-specific

Yes. It’s a question of what goes in the standard, what goes in an application profile and what is left to local interpretation.

Regarding your specific examples:

  • Por20349 - HSDS - US and UK Row 88 proposes an “external_identifier” which UK guidance or a UK public sector application profile might require to be a UPRN (as mandated by government)
  • I think we all see Open Referral as taxonomy agnostic but again an application profile might mandate a specific taxonomy like AIRS or the LGA’s service types in specific cases. Certainly, UK users who want to combine feeds from multiple sources want consistent use of taxonomies across those feeds.