Background
-
What types of evidence for effectiveness and value do Swedish public sector authorities require from bidders when procuring health and welfare technologies?
-
Are their differences in bidder attributes and procurement outcomes for those that require evidence for effectiveness and value compared to those that do not?
Methods
Inclusion criteria
Exclusion criteria
Search strategy
Search strings
-
English: welfare tech*; digital health tech*; digital health*; e-health*, safety alarm*; GPS-alarm*; security camera*; night camera*; nocturnal camera*; digital nocturnal surveillance; self-monitor*; remote monitor*; door alarm*; floor alarm*; absence alarm*; digital care; remote meeting*; digital lock*; robot*; incontinence sensor*; medication reminder*; medication alarm*; medication dispenser*; digital game*; digital rehab*; digital training*; smart*; health app*; digital activation*; epilepsy alarm*; seizure alarm*; digital learning*; digital education*; digital competency*;
-
Swedish: välfärdsteknik*; digital hälsoteknik*; digital hälsa; e-hälsa; trygghetslarm; GPS-larm; tillsynskamera; nattkamera; digital nattillsyn; egenmonitorering; fjärrmonitorering; larmmattor; dörrlarm; rörelselarm; avvikelselarm*; digital vård*; distansmöte*; digital* lås; robot*; inkontinens sensor*; medicinpåminnare*; läkemedelspåminnare*; läkemedel dispens*medicinlarm*; läkemedelslarm*; läkemedelsautomat*; läkemedelsdispenser*; digitalt spel*; digital rehab*; digital träning*; smarta*; hälsoapp*; digital aktivering*; epilepsilarm*; digital *lärning; digital *läromedel; digital kompetensutveckling
Request for tender selection process
Announcement screening
Request for tender documentation screening
Content screening for evidence-related terminology
Content confirmation for evidence-related criteria
-
Clear evaluation- or awarding criteria (in Swedish respectively known as utvärderingskriterier or tilldelningskriterier, often in the form of “shall” or “should” requirements) were used in the tendering documentation regarding evidence for effects or value, consisting of collected or objective data, a certification that is only achieved upon provision of such data, or similar and
-
the requirement was specific to the HWT intervention in the actual request for tender.
-
the requirement was solely related to the bidder’s overarching organisation (e.g. that the organisation is ISO-certified or uses a quality management system); or
-
the requirement is solely related to administration of the intervention (e.g. availability, delivery, user education); or
-
the requirement did not address a defined outcome (e.g. “improve effectiveness” without stating what that effectiveness involves or means for the organisation); or
-
there was no request for documentation, certification or data to demonstrate fulfilment of the requirement (e.g. a simple “Yes/No” response sufficed as fulfilling the requirement).
Data extraction
Outcomes of interest
-
the prevalence and type of terminology in the tendering documentation related to evidence for effects and/or value
-
the prevalence and type of evidence-related criteria when evaluating bids
-
the attributes of the procuring organisation, bidders, type of HWT and procurement result in relation to the above outcomes
Results
Type of procuring organisation | HWT type | Use of evidence-related criteria |
---|---|---|
Small municipalities (27 requests for tenders) | Automated pharmaceutical dispensers/robots (4) Combined HWT systems/packages (3) Digital locks/entry systems (3) Digital/GPS safety alarm with or without response chain (17) | 14.8% (4) |
Medium municipalities (37 requests for tenders) | Automated pharmaceutical dispensers/robots (7) Combined HWT systems/packages (6) Digital cognitive support (1) Digital locks/entry systems (4) Digital rehabilitation aids (1) Digital/GPS safety alarm with or without response chain (12) Digital surveillance (4) Digital therapy animals/robots (1) Digital training aid (1) | 16.24% (6) |
Large municipalities (2 requests for tenders) | Digital locks/entry systems (1) Digital/GPS safety alarm with or without response chain (1) | 0 |
Small regions (2 requests for tenders) | Digital cognitive support (1) Digital surveillance (1) | 0 |
Medium regions (5 requests for tenders) | Automated pharmaceutical dispensers/robots (1) Digital epilepsy/seizure alarm (1) Digital surveillance (1) Self-monitoring technologies (1) | 40% (2) |
Large regions (4 requests for tenders) | Automated pharmaceutical dispensers/robots (2) Combined HWT systems/packages (1) Digital/GPS safety alarm with or without response chain (1) | 50% (2) |
Othera (13 requests for tenders) | Automated pharmaceutical dispensers/robots (3) Digital locks/entry systems (2) Digital/GPS safety alarm with or without response chain (5) Digital surveillance (2) Self-monitoring technologies (1) | 14.2% (2) |
HWT type and year of request | Type of procuring organisationa and method | Tender value, EUR (max. Contract length) | Procurement outcomeb (number of bidders) | Evidence-related criteria |
---|---|---|---|---|
Digital/GPS safety alarm, 2021 | Small municipality, open procurement | €155,000 (8 years) | Awarded to 1 SME (3 bidders) | A system description must be provided as proof that the requested functions can be delivered; this will be assessed by an evaluation group. |
Digital/GPS safety alarm, 2021 | Small municipality, open procurement | €1,133,950 (8 years) | Awarded to 1 SME (4 bidders) | A system description must be provided as proof that the requested functions can be delivered; this will be assessed by an evaluation group. |
Self-monitoring technologies, 2021 | Medium region, simplified procurement | not available (10 years) | Appealed; Awarded to 1 MNC (bidders not available) | Medical personal will assess if the monitoring solution described is in line with scientific studies and national guidelines. |
Self-monitoring technologies, 2021 | Large region, negotiated procurement | €15,342,800 (7 years) | Ongoing | The provider’s solution must meet MDR 2017/745 CE standard of conformity alternatively MDD 93/42 during the entire contract period. The user must be able to receive evidence-based, or alternatively region-formulated, self-care guidance to improve treatment. |
Automated pharmaceutical dispensers/robots, 2020 | Medium municipality, simplified procurement | €295,800 (4 years) | Awarded to 1 SME (1 bidder) | The bidder must provide a CE standard of conformity document, a clinical study (brief report) and a clinical test report to show that the requirements placed on the product/service are met |
Self-monitoring technologies, 2020 | Medium region, open procurement | €1,045,100 (4.5 years) | Awarded to 1 MNC (6 bidders) | Bidders must be able to provide a certificate showing that the product meets the CE standard of conformity and MDR 2017-745 for those aspects that involve medical technical products. |
Digital locks/entry systems, 2020 | Medium municipality, open procurement | €315,700 (7 years) | Appealed; Awarded to 1 SME (3 bidders) | Product must meet industry security standard for class S3 locks and provide proof of certification from a nationally accredited third-party certification institute |
Digital/GPS safety alarm, 2020 | Medium municipality, open procurement | €2,395,900 (8 years) | Awarded to 2 SME (5 bidders) | Proof that the alarm meets a standard IP-classification; documentation confirming this is required. |
Digital/GPS safety alarm, 2020 | Member-owned corporation, open procurement | €158,739,550 (4 years) | Ongoing | Proof that the alarm to be worn does not contain specific chemicals; a signed assurance from the manufacturer, a chemical analysis protocol, or a full declaration of contents must be provided. Proof that the alarm communicates via an open protocol that meets a technical standard; for example, a test protocol from an external testing institute can be provided. |
Digital/GPS safety alarm, 2020 | Medium municipality, simplified procurement | €31,335 (13 years) | Awarded to 1 SME (2 bidders) | Products that fall under the regulations for CE standard of conformity must meet the requirements stated in the actual CE standard of conformity. Documentation that shows that these requirements are met must be available upon request. |
Digital locks/entry systems, 2020 | Medium municipality; Regionally owned corporation | € 104,700 (4 years) | Awarded to 1 SME (1 bidder) | The lock solution and the key safe must be approved and certified by the national association according to standard SS3522 / SSFN-024 class 3 or higher, or a similar certificate. The certificate must be included in the bid. |
Digital/GPS safety alarm, 2019 | Small municipality, open procurement | € 1,577,600 (4 years) | Appealed, no winner (3 bidders) | Regarding communication frequencies and standards, the equipment that falls under the directive for the CE standard of conformity must at the time of delivery meet the requirements for CE standard of conformity |
Automated pharmaceutical dispensers/robots, 2019 | Small municipality, open procurement | € 399,650 (4 years) | Awarded to 1 SME (2 bidders) | All products shall meet the CE standard of conformity, the national law on medical technical products (SFS 1993:584) and the national standard SS-EN 980 |
Automated pharmaceutical dispensers/robots, 2018 | Medium municipality, direct procurement | not available (0.5 years) | Awarded to 1 SME (1 bidder) | Requirement that the product should have test documentation and a valid CE standard of conformity; bidder should describe how they meet these requirements. |
Digital locks/entry systems, 2018 | Medium municipality, open procurement | € 957,800 (4 years) | Awarded to 1 SME (1 bidder) | Product must meet industry security standard for class 3 locks, and proof of certification must be provided in the bid |
Digital/GPS safety alarm, 2017 | Large region, open procurement | €973,200 (4 years) | Awarded to 3 SME (4 bidders) | Products must meet CE-standard of conformity according to MDD |
Use of evidence-related criteria | Average number of bidders | Procurement outcomea | Average max. Contract length | Type of procuring organisation |
---|---|---|---|---|
Yes (n = 14) a | 3.8 | Awarded: 78.6% Awarded after appeal: 14.3% Appealed, not awarded: 7.1% | 3.8 years | Small municipality: 28.5% Medium municipality: 42.8% Large municipality: 0% Small region: 0% Medium region: 7.1% Large region: 7.1% Other consortium: 14.3% |
No (n = 70) a | 3.9 | Awarded: 92.9% Appealed, not awarded: 7.1% | 3.9 years | Small municipality: 32.9% Medium municipality: 44.3% Large municipality: 2.9% Small region: 2.9% Medium region: 5.7% Large region: 2.9% Other consortium: 14.3% |
Discussion
Recommendations for stakeholders
-
National and/or regional policy makers should task appropriate authorities to develop and oversee a framework for evidence based HWT use, to support public sector procurers and developers achieve a common understanding of evidence use. The MDR should be a foundation in this framework, but also provide guidance adjustable to local conditions and user preferences.
-
Decision makers that initiate procurement processes should clearly define intentions and outcomes that matter – such as health, welfare, and efficiency in operational care delivery outcomes - for any HWT application, and in terms of the local conditions and known or expected user preferences. These intentions and outcomes will form the core of what pre-existing evidence is necessary, and what evidence needs to be generated during the lifecycle of the HWT.
-
Procurement officials must establish an evidence-based procurement process based on the intentions and outcomes prioritized by decision makers. Input should be obtained from researchers, developers and users when establishing this process. The resulting process should include a list of adaptable criteria for what evidence must be provided by bidders and how, as well as criteria for real world evidence generation after purchase and implementation. Evaluation of bids should also place significant weight on assessing such evidence, and contracts with winning bidders should include clauses regarding continual evidence generation as a condition for contract fidelity and/or extension. The use of MDR as a standard criterion should be vastly increased.
-
For HWT developers, evidence for application effectiveness as well as real world evidence generation after implementation are now engrained in the EU MDR. High-quality continual evidence generation should be planned for when developing applications and be adaptable to expected purchasers’ local conditions and needs and user preferences. For small- and medium-sized enterprises, partnership with both academia and presumptive purchasers and their users to assist in meaningful data collection and evidence summarization may be necessary in many cases.
-
Academia can play a significant role in creating a neutral place where stakeholders on the demand and supply sides can meet for mutual learning, sharing and co-creation of both innovations and evidence generation for HWT.