Appin was contracted by a large public sector client to work with their construction “arm” to establish a “Source List” of three (3) pre-qualified controls vendors who would be entitled to bid on work at pre-defined locations across Canada for the next 5 years. In addition to qualifying these vendors for the Source List, the process was to create a score for each vendor that would represent 40% of their total score on a “Best Value” assessment whenever they bid on a Source List job.
Appin was contracted by a large public sector client to work with their construction “arm” to establish a “Source List” of three (3) pre-qualified controls vendors who would be entitled to bid on work at pre-defined locations across Canada for the next 5 years. In addition to qualifying these vendors for the Source List, the process was to create a score for each vendor that would represent 40% of their total score on a “Best Value” assessment whenever they bid on a Source List job.
The process began with a thorough review of Appin’s Section 25 Specification and the latest BACnet standard by technical experts from both the client and Appin. This review created several lists of features that proponents would have to report on during the process. The first of these lists, Worksheet 1, consisted of 64 general questions about the vendor and features of their products that were considered mandatory. Schedule 1 is a mandatory BACnet Functionality Check list where information is required on specific BACnet functions of the Operator Workstation, Building Controller and Advanced Application Controller. All systems from all vendors would be required to comply with the requirements of all of the features identified in this Schedule. The next phase of the process was to identify, evaluate and establish scores for items that were not mandatory. This resulted in the creation of Worksheet 2 “Scored Technical Elements” and Schedule 2 “Scored BACnet Functionality Checklist”. These completed Worksheets and Schedules would form the basis of the technical score portion of the evaluation.
Forms were then created to enable an evaluation of the Dealership capacity and capability for each proponent in the regions where the various locations to be served are located. Several of these are quite remote from large population centres so the proximity of qualified service personnel and the ability to respond to the need for a site service visit within a reasonable time frame had to be assessed as part of the qualification process. A scoring system was set up to enable the dealership capability score to be combined with the technical score for the final assessment of each proponent.
Request for Qualifications (RFQ) that included all the worksheets, schedules and other evaluation forms was posted and all interested vendors had the opportunity to ask questions and comment on the items listed in the Worksheets and Schedules and to participate in discussions on the process. This resulted in several “mandatory” items being moved to “scored” and some adjustments to the scoring.
Bids closed and the evaluation process began. The technical submissions were reviewed first with an initial check of the mandatory items to ensure that all proponents could meet all the mandatory items identified in Worksheet 1 and Schedule 1. The proponents’ responses were all checked against the supporting documentation they had provided and, where necessary, additional information was requested to confirm their claim of compliance. Those not able to comply with the mandatory requirements were eliminated from the selection process and the balance of their documentation was not reviewed. The same review was then performed on the scored items and on the information provided on dealership capacity. Some scored items did not have supporting documentation and the proponents were asked to provide evidence of their ability to perform at the demonstration that represented the second part of the evaluation process.
The “Interoperability Demonstration” was an important part of the evaluation. Technical staff from the vendors, the client and Appin spent 2 days in a secure facility demonstrating specific pre-determined features of their equipment and providing practical demonstrations of some of the Scored Items they claimed to support that were not well documented. Technical staff from all vendors set up their equipment in one large room with connections to the secure system. They performed several demonstrations which included generating points and trending their own systems and those from the other proponents. All systems were crashed and brought back on line to assess the “worst case” scenario. Bandwidth usage was measured and security issues were checked by technicians from provided by the client. Much of the demonstration was performed with the monitor information projected onto screens so that evaluators and observers could better assess the results.
While the technical demonstrations were underway, confidential interviews were carried out by the evaluation team with representatives from each vendor. These interviews were designed to clarify any issues that had been uncovered in assessing and scoring the dealership capability and capacity.
Following the Interoperability Demonstration, some additional information was requested from the vendors which included screen shots and data collected during the demonstration and information on dealerships ability to meet client security requirements. The evaluation team then made whatever final score adjustments were necessary and created spreadsheets of the mandatory and scored technical items and the dealership capacity in order to develop the final composite score used to identify the successful vendors for the Source List. Successful vendors have the opportunity to increase their technical score annually on the anniversary of the competition as they make equipment improvements and advances and several of them have taken advantage of this opportunity. Claims of improvements and advances are checked with the same rigor as used for the initial scoring process.