Set Query Syntax

Sets are collections of data from a previous query that can be used in other searches. When a set is created, all duplicated records are expunged. Set generation forces a query to run to the end and can only be run on Boolean queries. Faceted, ranked, or weighted queries do not allow for the creation of sets; however, once a set has been created, it is possible to use the set to create facets. Sets can be run on queries that span multiple data stores. Using previously created sets as the search base in other queries can greatly speed a search because the number of records for the server to search is greatly limited.

Sets can be created on a system level and shared by an administrator or by individual users for their individual use.

When a user creates a set, each set is stored by name in the underlying storage for that user account. To recall an existing set, the user must simply add SET:set_name to the Boolean expression of the query.

Parameters

• outputset=<true, name(field), description>
• SET:<name>
• usesession (used only when doing a URL search)

At this time there is no way for the user to view a list of those sets he or she has created or been given access to by an administrator. Administrators with access to Set Definitions can see sets created and shared there.

Using IMAT Solutions Report Builder

Creating a Set

When creating a set using the IMAT Solutions compound report builder, use the outputset parameter within the query. Use outputset=True (capitalization is important) to create a random outputset. This method is suggested if the set is only intended for use within the same report and not for use in a separate report or query at a later time. The server creates the set data frame to the variable, my_set, in the following example. Once the data frame has been created, it can be treated as any other data frame.

my_set=get_hits('()asthma', fields=CLINICAL_REPORTS_FIELDS, store="1,3,5", outputset=True)

Assign a string to the outputset parameter to assign a name to a set that can be used at a later time, either within the same report or in another report. Underscores or camel case may be used to distinguish between words in the set name.

If you choose to enter a limit as part of your query, know that the set includes all data that match the criteria of the query rather than only the number of records you set as a limit. For example, if there are 1 million records in the system, if 1,000 of those records meet the set requirements, and if you put a limit of 10 on the query creating the set, then the set still contains 1,000 records even if you only see 10 of the results. This means that when you run another query using the results from the set, you are querying 1,000 records.

asthmaDF = get_hits('()asthma', fields=CLINICAL_REPORTS_FIELDS, store='Records', outputset="my_set(NREF)")

Below are the formats for naming sets. If querying records, you need only include the set name; however, if you are creating sets for a quick table field, you must include the field name.

• NREF: record-number-centric sets
  outputset="MRN_set"
• PatientIDN: patient-centric sets
  outputset="acoPopulation(PatientIDN)"
  Note:

  In order for sets to work with the PatientIDN field, stores must be properly flagged for the Quick Table before feeding records.
  See set functions created for the ease of combining and creating sets for viewing and analyzing results.

Using a Set

Compound Reports

After a set has been created, it can be used in other queries either within the same report or in other reports. To call a set and use it in other queries, follow the example below. Notice the use of SET:

my_query=get_hits('()SET:my_set n.age:[30 TO 40]', store='Records', fields='extract.patient_id') #This searches all the records from my_set and applies the additional age filter

When using a set that was written using the PatientIDN field (or other quick table field), you must include the Boolean operator FILTER.

my_query=get_hits('()FILTER(PatientIDN:(SET:__patients__))', store='Records', fields='extract.patient_id')

Note that when using the Report Builder, it is unnecessary to use usesession when using sets.

See set functions created for the ease of combining and creating sets for viewing and analyzing results.

Advanced Users

To target a set under a particular session, put target session name in front of the set name in the query expression term. For example, you might have a session id of group_testing and a SET name of test so the would become q=()SET:group_testing/test.

SET:targetSessionName/setName

The targetSessionName must appear in the list of session IDs. Note that the userSessionName is always listed first in the sessionid list.

sessionid=userSessionName,targetSessionName, ...

If you want to combine multiple sets that have the same name from different sessions, use the keyword UNION. This will widen the hit list like using an OR expression.

SET:UNION/setName

If you don’t qualify the setName in the SET expression, each session listed under &sessiondid= will be explored in the order listed. The first session that contains the setName will be used. This is the standard behavior for SET:setName.

Single Reports

To use sets in the single query report builder, they must be created by an administrator using Set Definitions. Once the set has been created, enter the set using proper set syntax. Once again, you must use FILTER if using a PatientIDN (or other quick table) set. Enter the set in the code section provided in the single report builder.

()FILTER(PatientIDN:(SET:__patients__))

()SET:MRN_set

Outputset

To output a set of refs, use one of the methods of starting a set, and the parameter outputset. Using outputset=true outputs a random set.

https://<appliance_ip>/search?q=(computers)&outputset=true

Outputset also takes a user-specified name.

https://<appliance_ip>/search?q=()migraine&outputset=poorHead(NREF) # You must include either NREF or PatientIDN

Sets can also have a description of less than 100 characters of what is in the set.

https://<appliance_ip>//search?q=(migraine)&outputset=poor_head(NREF),list of all migraine sufferers

SET and Usesession

Once a set has been generated, it can be used as part of a query. Use SET:setName as one of the Boolean terms, and use &usesession as a parameter. Usesession is only required when doing a search using the search bar of your browser.

https://<appliance_ip>/search?q=(LastName:smith AND SET:S1 AND SET:S2)&usesession

Sets can also be used as part of a query that outputs a third set:

https://<appliance_ip>/search?q=(SET:S1 OR SET:S2)&outputset=S3&usesession

Additional examples of how to create and use sets are below:

Make a set of all records from Timpanogos hospital:

https://<appliance_ip>/search?q=()facility:“Timpanogos Hospital”&outputset=TimpRecords(NREF)

Find all records designating patients age 70 and older at Timpanogos hospital:

https://<appliance_ip>/search?q=()age:[70 TO *]  SET:TimpRecords&usesession

Make a set of patients with diabetes (PatientIDN is found in the Quick Table):

https://<appliance_ip>/search?q=()diagnosis:diabetes &outputset=diabetics(PatientIDN)

Use the FILTER function with PatientIDN sets. Find all records designating patients age 70 and older that are diabetics:

https://<appliance_ip>/search?q=()age:[70 TO *] FILTER(PatientIDN:(SET:Diabetics))&usesession

Find records of all diabetics at Timpanogos hospital:

https://<appliance_ip>/search?q=()SET:TimpRecords FILTER(PatientIDN:(SET:Diabetics))&usesession

Faceting on SETs

While a set cannot be created on a query that has facets as part of the query, a set can be used with facets after it has been generated.

https://<appliance_ip>/search?q=(SET:S1 OR SET:S2)&facets=lastmodified:range(begin:-1years,end:today,before:true,gap:-3 month),contentlength:range(begin:0,end:25M,after:true,gap:5M)&usesession 

Using Admin: Set Definitions

Users with the correct permissions can create sets using the Set Definitions tool. Using Set Definitions makes it possible to share the sets among users and groups. From here you can also rerun sets if data becomes outdated due to new records, changed records, or changed store attributes.