RFQ-25-401 - Energy Code Compliance Evaluation Support

REQUEST FOR QUALIFICATIONS

Energy Code Compliance

Evaluation Support

RFQ-25-401

State of California

California Energy Commission

May 2026

ATTACHMENTS

I. INTRODUCTION

Background Summary

The Warren-Alquist Act, Public Resources Code (PRC) section 25000 et seq., established the California Energy Commission (CEC) as California's primary energy policy and planning agency. It also mandated that the CEC adopt and periodically update the Building Energy Efficiency Standards, which include building efficiency requirements in the Energy Code (California Code of Regulations, Title 24, Part 6) and voluntary building efficiency standards in the California Green Building Standards Code (California Code of Regulations, Title 24, Part 11).

The CEC serves as the state of California's (State) primary energy policy and planning agency. A core responsibility of the CEC is the adoption and implementation of California's Building Energy Efficiency Standards which apply to newly constructed single-family residential buildings, multifamily buildings, nonresidential buildings, and certain processes, as well as additions and alterations to these buildings across California.

The overarching statutory requirements of the Energy Code are to reduce wasteful, uneconomic, inefficient, or unnecessary energy use, and its goals are to lower energy costs for Californians, improve energy affordability, advance energy efficiency, and contribute to California's climate action goals. The 2022 Energy Code is estimated to provide California with $8.8 billion in statewide benefits over its lifetime, and the 2025 Energy Code is estimated to provide $4.8 billion in statewide benefits over its lifetime. Since inception, energy efficiency standards for both buildings and appliances have saved Californians over $200 billion in energy costs. However, the realization of these projected energy savings is contingent upon all design and construction activities complying with State regulations, including the Energy Code.

The Challenge: Lack of Comprehensive Compliance Data

The State currently faces significant challenges validating statewide compliance rates due to the absence of standardized, representative, and field-verified compliance data across building types and jurisdictions. This includes a scarcity of information on regional variations in compliance and the underlying causes of noncompliance. This data gap limits the CEC's ability to fully grasp the financial cost of noncompliance to the State, effectively implement State policy, and strategically allocate resources. The CEC published a report, "Energy Code Compliance Gap Analysis," which elaborates on these conditions by identifying challenges and proposing solutions to enhance the State's understanding of Energy Code compliance evaluation going forward.

Overarching Study's Principles

This Energy Code Study (also known as the Compliance Rates Study) is intended to produce a statewide snapshot of the current California Energy Code compliance practices and outcomes, reflecting conditions at the time of data collection.

The study will focus on:

Observed compliance at design and construction stages.

Common pathways, barriers, and failure points of Energy Code implementation.

Variability across building types, jurisdictions, and compliance approaches.

Actionable insights to inform the compliance improvements strategies, workforce development, and future work.

Unlike program evaluation designs that assess pre and post implementation impacts, this study will not attempt to estimate changes over time or causal impacts of specific interventions, due to cost, feasibility, and burden consideration.

Understanding Compliance: Key Definitions

To establish a clear framework for compliance evaluation, CEC staff propose the following clear fundamental compliance definitions:

Process-Based Compliance: This addresses building construction projects that successfully obtain necessary permits, undergo the full permitting and inspection processes, and maintain accurate documentation from design through completion. It focuses on adherence to procedures and documentation but does not inherently guarantee that the project underachieves or overachieves the intended energy savings.

Energy-Based Compliance: This focuses on the actual outcomes, meaning the completed project performs at or above the energy efficiency levels intended by the Energy Code. Unlike process-based compliance, it does not strictly require all procedural steps to be completed.

Full compliance is achieved when a project satisfies both process-based and energy-based criteria.

While the State aims to improve full compliance, evaluating energy-based compliance presents a fundamental challenge because the Energy Code is a performance-based standard, as required by the Warren-Alquist Act. This structure allows multiple compliance pathways and design tradeoffs for each building, resulting in highly project-specific compliance outcomes. In addition, many Energy Code measures are only observable or verifiable during specific construction phases, once a building is occupied. Access to intermediate measures (such as envelope assemblies, concealed systems, or installed controls) is limited or infeasible without disruption. As a result, statistically representative field evaluation of energy-based compliance-particularly at a statewide scale-can become complex, costly, and resource-intensive.

Current Landscape of Compliance Evaluation in California

Previous studies in California have presented divergent findings regarding Energy Code compliance and do not necessarily provide a comprehensive statewide assessment.

Investor-Owned Utility (IOU) Programs: Evaluations of IOU Codes and Standards (C&S) Building Code Advocacy programs, overseen by the California Public Utilities Commission (CPUC), have historically shown high "compliance" or "Energy Saving Factor" rates (over ninety percent [90%]). However, these studies were primarily designed to assess program-caused energy savings attributable to the IOU programs, rather than evaluating overall statewide adherence to the Energy Code. This distinction means their findings may not represent the broader state of code compliance of buildings in the State. In addition, prior IOU evaluations have therefore focused primarily on post-construction or readily observable measures, which are generally found to be satisfactory but do not capture the full scope of intermediate or construction-phase compliance.

Regional Energy Networks (RENs): Some RENs, authorized by the CPUC, have conducted regional research. For instance, the 2015 Bay Area REN (BayREN) Permit Resource Opportunity Program (PROP) report revealed that only sixteen percent (16%) of projects in the Bay Area included complete compliance documentation, indicating a significant gap in understanding and adherence to the compliance process.

Unpermitted Construction: A significant portion of construction activity occurs outside the building permitting process, representing an understudied segment. Studies indicate low permitting rates for certain activities, such as heating, ventilation, and air conditioning (HVAC) changeouts (eight to twenty-nine percent [8-29%] permitted) and newly constructed accessory dwelling units (ADUs) in some cities (around twenty-five percent [25%] permitted). These low permitting rates suggest a lack of process-based compliance and potential negative impacts on energy-based compliance. The true extent and impact of unpermitted projects on noncompliance and associated costs to the State remain unclear due to significant data gaps.

Unpermitted construction occurs across a range of project types and may be performed by both licensed contractors and unlicensed installers. Licensed contractors are generally required to obtain permits for construction activities subject to local building regulations; however, in practice, work often is completed without permits. Unpermitted construction is also often completed by unlicensed installers performing work that requires licenses. Licensed contractors who do work under permits often view installers who do work without permits or without licenses as unfair competition, being underbid, potentially resulting from less attention to construction quality or compliance with building codes. By its nature, unpermitted construction operates outside of oversight and may not be visible to building departments or the California Contractors State License Board (CSLB). CSLB protects consumers by regulating the construction industry through policies that promote the health, safety, and general welfare of the public in matters relating to construction. Building owners that knowingly authorized the work to be done without permits or by unlicensed installers may not be willing to disclose or participate freely in field research regarding compliance with the Energy Code.

The CEC endeavors to evaluate compliance for both permitted and unpermitted projects, including work performed by both licensed contractors and unlicensed installers. The CEC recognizes the complexity and difficulty of studying unpermitted work. The objective is to produce compliance estimates that are as valid, accurate, and representative as possible across these market segments. The evaluation is expected to encounter challenges in recruiting participating buildings for unpermitted and unlicensed cases. This may result in difficulty in avoiding selection bias that would limit the validity of findings for unpermitted cases.

The Energy Code has long established requirements for system sizing, and for system quality installation (including duct sealing, refrigerant charge testing, airflow testing, and fan watt draw). Poor installation quality related to these requirements can compromise energy efficiency, particularly for air conditioning equipment including heat pumps. Several of these key requirements require field testing of equipment to demonstrate compliance. The performance of heat pumps, which are critical to accomplishing State building decarbonization and climate change goals, can be especially negatively impacted by continued failure to do work under permits, not only for space cooling but also for space heating. Residential water heater replacements also may frequently be completed without permits or by unlicensed installers. The performance of water heaters, particularly heat pump water heaters, may also be negatively impacted by failure to comply with system sizing and airflow requirements in the Energy Code. Failure to do such water heater or HVAC replacements without permits or without contractor licenses may also result in health and safety risks.

Seven Building Categories

This solicitation aims to address key gaps identified in the Gap Analysis report, specifically supporting the CEC to execute tailored field studies for the following seven (7) building categories:

Existing single-family residential buildings.

Existing multifamily residential buildings.

Existing nonresidential buildings.

Newly constructed single-family residential buildings.

Newly constructed multifamily residential buildings.

Covered processes.

The goal is to ensure accurate compliance assessment across seven (7) distinct building categories. The primary differentiators among them relate to the complexity of the building structure, the accessibility and nature of compliance documentation, the prevalence of the unpermitted market, and the appropriate tools for energy modeling. Because each building category has distinct characteristics, category-specific methodologies are required to achieve a comprehensive and defensible evaluation.

This solicitation seeks to identify qualified contractors to support the CEC in conducting comprehensive California Energy Code compliance evaluations and developing robust tracking systems, aligning both State and federal objectives for energy efficiency and effective building code implementation.

Purpose of this RFQ

The purpose of this Request for Qualifications (RFQ) is to select a Prime Contractor to lead a team of professional architectural and engineering consultants to provide technical support for:

The California Energy Code compliance evaluation program, including the design and execution of methodologically defensible evaluations to address critical statewide data gaps and improve the CEC's understanding of both process-based and energy-based compliance.

Improving the State's intelligence regarding Energy Code compliance through evidence-based identification of systemic challenges, regional variations, and root causes of noncompliance, informed by empirical field data rather than program attribution or advocacy-focused analyses.

Establishing a technical foundation for future statewide Energy Code compliance rates analyses, including the development and validation of evaluation methodologies, sampling frameworks, data collection instruments, and analytical approaches that can be replicated and scaled in subsequent studies.

Supporting implementation of recommendations from the "California Energy Code Compliance Gap Analysis" staff report by providing technical evaluation, feasibility assessment, and empirical evidence to inform enforcement strategies, compliance improvement priorities, and future policy development.

Conducting comprehensive field studies across defined building categories and compliance definitions, recognizing the performance-based nature of the Energy Code, the existence of multiple compliance pathways, construction-phase access limitations, and the inherent complexity of evaluating energy-based outcomes at scale.

Identifying and supporting technically grounded compliance implementation solutions, including evaluation-informed recommendations, pilot concepts, workforce-related technical assistance, and implementation-ready tools or frameworks that enable State, regional, and local partners to improve Energy Code compliance outcomes based on documented findings from the evaluation activities.

Key Activities and Dates

Key activities including dates and times for this RFQ are presented below. An addendum will be released if the dates change for the asterisked (*) activities.

Times listed are Pacific Standard Time or Pacific Daylight Time, whichever is being observed.

Available Funding

There is a maximum of up to $11,500,000 available to fund the three (3) year contract resulting from this RFQ. This is an hourly rate plus cost reimbursement contract with a ceiling on the total contract amount.

Funding for the agreement is from the CEC's Inflation Reduction Act (IRA) Building Energy Codes Award from the United States Department of Energy's (DOE) Office of State and Community Energy Programs (SCEP). The CEC reserves the right to reduce the contract amount to an amount deemed appropriate in the event the any of the grant awards from a federal agency of the United States (US) that provide funding for this agreement, do not appropriate sufficient funds for the work identified. In this event, the CEC shall have the option to either: cancel this agreement with no liability occurring to the State, or the CEC Contract Agreement Manager (CAM) and Contractor shall meet and reach agreement on a reduced Scope of Work commensurate with the level of available funding. See the terms and conditions in the Standard Agreement Example (Attachment 6) for more information.

Eligible Firms

This is an open solicitation for public and private entities. Each agreement resulting from this solicitation includes terms and conditions that set forth the Contractor's rights and responsibilities. The University of California, California State University, or DOE National Laboratories must use either the standard or the pre-negotiated terms and conditions on the at (http://www.dgs.ca.gov/OLS/Resources). All other entities must agree to use the Standard Terms and Conditions (Attachment 6). The CEC will not award agreements to non-complying entities. The CEC reserves the right to modify the terms and conditions prior to executing agreements.

All corporations, limited liability companies (LLCs), limited partnerships (LPs), and limited liability partnerships (LLPs) that conduct intrastate business in California are required to be registered and in good standing with the California Secretary of State (SOS) prior to a project being recommended for approval at a CEC business meeting. If not currently registered with the SOS, applicants are encouraged to contact the SOS Office as soon as possible to avoid potential delays in beginning the proposed project(s) (should the application be successful).

For more information, visit the at (http://www.sos.ca.gov/). Sole proprietors using a fictitious business name must be registered with the appropriate county and provide evidence of registration to the CEC prior to their project being recommended for approval at a CEC business meeting.

Retainer Contract

Any contract awarded as a result of this RFQ will be a no-fee "retainer" contract. The selected Contractor will be held on retainer and will be assigned work via work authorizations. Work authorizations will be assigned by expertise, or project workload. The CEC makes no guarantee that any or all of the funds will be assigned in any given year.

Pre-Bid Conference

There will be one (1) Pre-Bid Conference. Participation in this meeting is optional but encouraged. The Pre-Bid Conference will be held remotely through Zoom at the date, time, and location listed below.

Date: Tuesday, May 19, 2026

Time: 1:30 p.m. - 3:00 p.m.

(Pacific Standard Time or Pacific Daylight Time, whichever is being observed)

Remote Access Only Via Zoom

Applicants may attend the workshop via the Internet (Zoom, see instructions below), or via conference call on the date and at the time listed below. Please contact the Commission Agreement Officer (CAO) listed on page 15 or refer to the at (https://www.energy.ca.gov/funding-opportunities/solicitations) to confirm the date and time.

Zoom Instructions:

Zoom is the CEC's online meeting service. When attending remotely, presentations will appear on your computer/laptop/mobile device screen, and audio may be heard via the device or telephone. Please be aware that the Zoom meeting will be recorded and posted to the CEC website.

To join the Pre-Bid Conference, go to the , which is located at . Participants must register to attend. After registering, you will receive a confirmation email containing information about joining the meeting. You may also access the conference by going to the , which is located at (https://zoom.us/join) and enter the unique Meeting ID below. Select "join from your browser." Participants will then enter the meeting password below and their name. Participants will select the "Join" button.

Meeting ID: 832 9739 6601

Meeting Password: 192979

Topic: Energy Code Compliance Evaluation Support Pre-Bid Conference

Telephone Access Only:

Call 1-888-475-4499 (Toll-Free) or 1-877-853-5257 (Toll-Free). When prompted, enter the meeting number above. International callers may select a number from the at (https://energy.zoom.us/u/abEf4RINDr). To comment, dial *9 to "raise your hand" and *6 to mute/unmute your phone line.

Access by Mobile Device:

Download the application from the at (https://energy.zoom.us/download).

Technical Support:

For assistance with problems or questions about joining or attending the meeting, please call Zoom Technical Support at 1-888-799-9666 extension 2. You may also contact the Public Advisor's Office by email at or by telephone at 1-916-269-9595.

: To determine whether your computer is compatible, visit (https://support.zoom.us/hc/en-us/articles/201362023-System-requirements-for-Windows-macOS-and-Linux).

If you have a disability and require assistance to participate, please contact Erica Rodriguez by email at or by telephone at 1-916-764-5705 at least five (5) days in advance of the Pre-Bid Conference.

Questions

During the RFQ process, questions of clarification about this RFQ must be directed to the CAO listed in the following Contact Information section. Potential Firms shall carefully examine the qualifications and specifications of this RFQ. You may ask questions at the Pre-Bid Conference, and you may submit written questions via electronic mail. However, all questions must be received by 5:00 p.m. on the date indicated in the Key Activities and Dates section.

The questions and answers will be posted on the at (https://www.energy.ca.gov/funding-opportunities/solicitations).

Any verbal communication with a CEC employee concerning this RFQ is not binding on the State and shall in no way alter a specification, term, or condition of the RFQ. Therefore, all communication should be directed in writing to the CAO assigned to the RFQ and listed below.

Contact Information

Diana Grady Commission Agreement Officer

California Energy Commission

715 P Street, 3rd Floor, MS-18

Sacramento, California 95814

Email:

Responses to This RFQ

Responses to this solicitation shall be in the form of a Statement of Qualifications (SOQ) according to the format described in this RFQ (See Section III: SOQ Format, Required Documents, and Delivery). The SOQ shall detail the Firm's qualifications to perform the tasks outlined in the Scope of Work.

Reference Documents

Firms responding to this RFQ may want to familiarize themselves with the following documents:

California Energy Code Compliance Gap Analysis Staff Report located at (https://www.energy.ca.gov/publications/2025/california-energy-code-compliance-gap-analysis).

located at (https://www.energy.ca.gov/programs-and-topics/programs/building-energy-efficiency-standards/2022-building-energy-efficiency-3).

located at (https://www.energy.ca.gov/programs-and-topics/programs/building-energy-efficiency-standards/2022-building-energy-efficiency-4).

located at (https://www.energy.ca.gov/publications/2022/2022-nonresidential-and-multifamily-compliance-manual-2022-building-energy).

located at (https://www.energy.ca.gov/programs-and-topics/programs/building-energy-efficiency-standards/2022-building-energy-efficiency-2).

located at (https://www.energy.ca.gov/publications/2022/2022-single-family-residential-compliance-manual-2022-building-energy-efficiency).

located at (https://www.energy.ca.gov/publications/2022/2022-single-family-residential-alternative-calculation-method-reference-manual).

located at (https://www.energy.ca.gov/publications/2022/2022-nonresidential-and-multifamily-alternative-calculation-method-reference).

located at (https://www.energy.ca.gov/programs-and-topics/programs/building-energy-efficiency-standards/2025-building-energy-efficiency).

located at (https://www.energy.ca.gov/programs-and-topics/programs/acceptance-test-technician-certification-provider-program).

located at (https://www.energy.ca.gov/programs-and-topics/programs/home-energy-rating-system-hers-program).

located at (https://www.energy.ca.gov/programs-and-topics/programs/energy-code-compliance-program).

located at (https://www.energy.ca.gov/funding-opportunities/funding-resources/formatting-reports-and-writing-style-consultants-california).

II. SCOPE OF WORK (SOW)

About This Section

In this section, the CEC describes the tasks the Firm (referred to as "Contractor" in the SOW) will be asked to perform under the direction of the CAM. This section also describes the work assignment process, deliverables, and due dates.

Purpose

The purpose of this agreement is for the Prime Contractor to lead a team of professional architectural and engineering consultants to provide technical support for:

The California Energy Code compliance evaluation program to include the following work areas: evaluation, market research, and data analysis. This includes conducting compliance evaluations, identifying streamlined approaches, and assisting the CEC with improving compliance. The work will significantly enhance the CEC's understanding of compliance rates, regional variations, and the root causes of noncompliance, informing policy and resource allocation.

Workforce development market research and program development recommendations related to improving Energy Code implementation.

Supporting compliance improvement pilot solutions directly with local partners.

Other related work that supports Energy Code compliance improvement.

Acronyms/Glossary

Specific acronyms and terms used throughout this SOW are defined as follows:

Work Authorizations

The agreement that results from this solicitation shall be conducted as a "Work Authorization" agreement. No work shall be undertaken unless authorized by the CAM through a specific written document called a "Work Authorization" (WA).

The CAM will prepare and issue the written WAs and shall set a maximum price, budget, and schedule for the work to be performed. The CAM will work, in consultation with the Contractor, to assign work to either the Contractor or a Subcontractor.

Retainer Contract

This is a no-fee "retainer" contract. The Contractor will be held on retainer and will be assigned work via WAs, which will be assigned by expertise or project workload. The CEC makes no guarantee that any or all the funds will be assigned in any given year.

No Work Guarantee

The CEC does not guarantee any minimum or maximum amount of work to the Prime Contractor or any Subcontractor under the agreement.

Workshops & Hearings

All workshops and hearings are sponsored, organized, and facilitated by the CEC. The CEC is responsible for any costs associated with a workshop or hearing. The Contractor shall provide labor only.

Incidental Services

Contractor shall provide incidental services to support the technical tasks that the CEC will undertake for Energy Code Compliance Evaluation Support in the general topic areas listed below. Technical Tasks 2 through 11 provide more detailed task activities for these areas:

Graphic Design/Document Support for reports and other deliverables related to the Energy Code.

Public Outreach and Communication/Marketing/Public Relations/Program Development necessary to complete the goals of this agreement.

Draft and Final Deliverables/Reports

The Contractor may be required to produce several iterations of draft deliverables to incorporate CEC's comments and edits. A deliverable is considered final when the CAM or Work Authorization Manager (WAM) indicates in writing that the deliverable is considered final. When creating reports, the Contractor shall use and follow, unless otherwise instructed in writing by the CAM or WAM, the following:

located at (https://www.energy.ca.gov/publications/2020/style-manual-fourth-edition-used-california-energy-commission-staff-lead).

located at (https://www.energy.ca.gov/media/2216).

The CEC typically requires the submission of products in an electronic format. If a hard copy deliverable is required, each final hard copy deliverable shall be delivered as one (1) original, reproducible, 8 12" by 11", camera-ready master in black ink, unless otherwise directed by the CAM or WAM. Illustrations and graphs shall be sized to fit an 8 12" by 11" page and readable if printed in black and white.

Ownership of Work Product

All deliverables, reports, data, memoranda, and other supporting documents developed under this SOW, whether completed or not, shall become the property of the CEC.

Electronic File Format

The Contractor shall submit all required data and documents as products under this agreement in an electronic file format that is fully editable and compatible with the CEC's software and Microsoft (MS) operating computing platforms, or with any other format approved by the CAM or WAM.

Unless otherwise specified by the CAM or WAM, the following describes the accepted formats of electronic data and documents provided to the CEC as agreement products and establishes the computer platforms, operating systems, and application versions that will be required to review and approve all application products.

Data sets shall be in MS Access or MS Excel file format (version 2016 or later).

PC-based text documents shall be in MS Word file format (version 2016 or later).

Documents intended for public distribution shall be in Adobe Portable Document Format (PDF) file format and shall comply with the accessibility requirements of of the federal Rehabilitation Act of 1973, which is located at (https://www.access-board.gov/about/law/ra.html#section-508-federal-electronic-and-information-technology). The Contractor shall also provide the native file format.

Project management documents shall be in a file format specified by the CAM or WAM.

Primary Tasks

The major categories of work are divided into the following tasks:

Task 1: Agreement Management

A maximum of ten percent (10%) of the total agreement budget will be allocated for this task. The Contractor will be required to perform contract management and administrative duties to manage the agreement. The Contractor shall also ensure sufficient time and attention is spent on this task by an experienced and qualified dedicated administrative staff person. At the discretion of the CAM, meetings, briefings, and discussions may be held via conference call, MS Teams, or Zoom.

The goal of this task is to provide for overall administrative management of the contract by the Contractor. In addition to the specific tasks below, the Contractor's Program Manager (PM) is responsible for directing the work performed by the Contractor Team to meet the objectives of the contract.

The PM is also responsible for ensuring the quality and timely delivery of all deliverables, both technical and administrative from the Contractor Team. The PM will be the primary point of contact for the Contractor Team and is responsible for oversight of all work under this contract. The PM is also responsible for managing all Subcontractor work, including ensuring quality products, enforcing Subcontractor agreement provisions, and in the event of failure of the Subcontractor to satisfactorily perform services, recommending solutions to resolve the problem.

The Contractor's responsibilities under this task include, but are not limited to, the following subtasks:

Subtask 1.1: Kick-Off Meeting

The goal of this subtask is to establish the lines of communication and procedures for implementing this agreement.

The Contractor shall:

Attend a "kick-off" meeting with the CAM and other appropriate CEC staff. The CAM will designate the specific location. The Contractor shall include its PM, Contracts Administrator, Accounting Officer, and others designated by the CAM in this meeting. The administrative and technical aspects of this agreement will be discussed at the meeting.

If necessary, prepare an updated Schedule of Deliverables based on the decisions made in the kick-off meeting.

Provide an Information Security Program Plan (ISPP) for CEC review and approval, as well as submit a signed ISPP Attestation Form confirming that Contractor has an ISPP that meets the minimum requirements as stated in State Administrative Manual (SAM) 5300 and any other applicable law.

Submit signed Non-Disclosure Agreement (NDA) Forms from Contractor and Subcontractor employees prior to the sharing of confidential information with the employees.

Ensure that all individuals employed by Contractor or a Subcontractor who will have access to confidential information take an annual security awareness training and submit the Employee Security Awareness Training Certificates to the CAM. Contractor and Subcontractors shall not invoice for the time spent attending security awareness training.

If necessary, provide the information necessary for the CAM to complete a Form 805 identifying any individuals designated as consultants.

Note that individuals designated as consultants under this contract must file a Form 700. The Contractor and Subcontractors shall not invoice for the time spent filing the Form 700. For additional information, see the terms and conditions in the Standard Agreement Example (Attachment 6).

The CAM shall:

Arrange the meeting including scheduling the date and time.

Provide an agenda to all potential meeting participants prior to the kick-off meeting.

Provide the ISPP Attestation Form.

Provide the NDA Form.

Complete the Form 805 (if necessary).

Contractor Subtask Deliverables:

An updated schedule of deliverables (if applicable).

ISPP and signed ISPP Attestation Form.

Signed NDAs.

Employee Security Awareness Training Certificates.

Form 700 (for individuals designated as consultants).

Subtask 1.2: Program Meetings and Briefings

The goal of this subtask is to ensure direct collaboration with CEC staff, staff of other public agencies, and participating external stakeholders throughout the completion of Tasks 2 through 11, and to disseminate information to all parties as needed.

The Contractor and Subcontractor(s) shall:

At the request of the CEC's CAM, be available for meetings or to provide written or verbal program briefings to the CEC's staff or others. The cost of meetings with local governments and public institutions will be included in each WA. The cost of meetings requested specifically by the Contractor shall be borne solely by the Contractor. At the discretion of the CAM, meetings, briefings, and discussions may be held via conference call, MS Teams, or Zoom. If requested by the CAM, prepare meeting notes to capture issues, action items, and feedback.

The CEC expects to hold no less than one (1) program briefing meeting per month.

Contractor Subtask Deliverables:

Meeting notes to capture issues, action items, and feedback (if applicable).

Subtask 1.3: Invoices

The goal of this subtask is to ensure accurate and timely payment for work performed under the contract.

The Contractor shall:

Prepare invoices for all reimbursable expenses incurred performing work under this agreement in compliance with the Exhibit B of the terms and conditions of the agreement.

Invoices shall be submitted with the same frequency as progress reports (Subtask 1.4).

Invoices must be submitted to the CEC's Accounting Office no later than fifteen (15) calendar days after the end of the monthly invoicing period.

The CAM shall:

Provide the format for invoices.

Contractor Subtask Deliverables:

Monthly invoices.

Subtask 1.4: Progress Reports

The goal of this subtask is to periodically verify that satisfactory and continued progress is made towards achieving the objectives of this agreement.

The Contractor shall:

Prepare monthly progress reports that align with the monthly invoices that summarize all agreement activities conducted by the Contractor for the monthly reporting period, including an assessment of the ability to complete the agreement within the current budget, Disabled Veteran Business Enterprise (DVBE) participation (if applicable), and any anticipated cost overruns. The Contractor shall provide the CAM a plan to remedy any anticipated cost overruns, as well as provide burn rate estimates to the CAM, upon request.

Each progress report is due no later than fifteen (15) calendar days after the end of the monthly reporting period.

The CAM shall:

Provide the format for the progress reports.

Contractor Subtask Deliverables:

Monthly progress reports.

Subtask 1.5: Management of WAs

WAs define the SOW, schedule of deliverables, and the project(s) budget. The goal of this subtask is to develop and manage all technical and budgetary aspects of WAs in accordance with the requirements of this agreement for work to be performed under Technical Tasks 2 through 11.

The Contractor shall:

At the direction of the CAM, assist the CEC in preparing the WAs in accordance with contract requirements.

Submit all required WA documents to the CAM.

Administer WAs.

Develop project schedules and adhere to the promised schedules for deliverable delivery to the CEC.

Provide audit and accounting services for all WAs.

Determine the fiscal status of each WA and the overall agreement.

Prevent cost overruns.

Provide oversight and first-level review of reports and documentation, and comment on the content of deliverables.

Ensure that final deliverables have been reviewed for technical accuracy, are ready for publication, and comply with all web accessibility requirements.

Adequately document all analyses so that the public can easily follow the logic of each analysis.

Monitor and track each WA and the overall agreement.

Provide updated WA project schedules, as needed, and determine if each WA is on schedule and deliverables are satisfactory.

Immediately report any significant variances affecting performance of WAs and recommend mitigation actions for consideration by the PM and CAM. Examples of significant variances include the inability to submit deliverables by key WA due dates, unavailability of key personnel that will affect timely submittal of deliverables, and key technical issues that would require change in scope, redirection of the effort, or discontinuation of the project.

Track the start, progress, and closure of each WA.

Coordinate with the CAM to close out completed WAs and remaining unallocated balances.

The CAM shall:

Provide the WA template and specify the required content.

Contractor Subtask Deliverables:

WA documents.

Subtask 1.6: Manage Subcontractors

The goal of this subtask is to manage Subcontractors' activities.

The Contractor shall:

At the direction of and in collaboration with the CAM, review and prepare WA scopes, deliverables, and budgets.

The Contractor is responsible for the quality of all Subcontractor work and for ensuring that Subcontractor work adheres to the promised schedule for deliverable delivery to the CEC.

Establish and maintain subcontract agreements.

Enforce subcontract provisions and manage Subcontractor activities in accordance with the agreement terms and conditions.

In the event of Subcontractor failure to perform, recommend solutions to resolve the problem.

When new Subcontractors are added, the Contractor shall ensure that: 1) The new Subcontractors comply with the terms and conditions of the agreement, and 2) Notify the CAM who will follow the CEC's process for adding or replacing Subcontractors. Subcontractors must be added to this agreement prior to beginning any work.

Subtask 1.7: Final Meeting

The goal of this subtask is to discuss closeout of this agreement and review the project.

The Contractor shall:

Meet with CEC staff prior to the term end date of this agreement. The CAM will designate the specific location. At the discretion of the CAM, meetings, briefings, and discussions may be held via conference call, MS Teams, or Zoom. The CAM and Contractor PM will attend this meeting. The CAM will determine any additional appropriate meeting participants. The administrative and technical aspects of agreement closeout will be discussed at the meeting.

Present findings, conclusions, and recommended next steps (if any) for the agreement.

Prepare a written document of meeting agreements and unresolved activities.

Prepare a schedule for completing the closeout activities for this agreement, based on determinations made within the meeting.

Contractor Subtask Deliverables:

Written documentation of meeting agreements.

Schedule for completing closeout activities.

Findings, conclusions, and recommendations.

Task 2: Technical Project Management

The goal of this task is to provide technical coordination and integration across all evaluation, sampling, field analysis, and reporting activities conducted under this agreement. The Contractor shall ensure that all analyses, tools, methodologies, and deliverables produced under this agreement meet the CEC technical standards, are developed in a transparent and responsive manner, and are delivered on schedule.

At the direction of the WAM, the Contractor shall manage and coordinate the technical work performed by the Contractor team and Subcontractors to ensure technical rigor, transparency, schedule adherence, alignment to CEC expectations, technical requirements, and federal requirements (see Standard Agreement Example, Attachment 6). The Contractor shall designate a Technical Project Manager who will be the primary technical point of contact and shall ensure rigorous technical quality, timely communication, and adherence to agreed-upon workflows.

In addition to the specified tasks below, the Contractor is responsible for ensuring that all technical products are fully documented, use CEC standard templates when available, and include underlying datasets, models, and tools necessary for independent review and validation.

The work in this task is expected to include, but not be limited to, the following:

Subtask 2.1: Technical Meeting and Coordination

The goal of this subtask is to ensure regular, proactive communication between the Contractor, CEC staff, Subcontractors, and external partners throughout all stages of the technical work. The CEC will initiate and engage with external interested party working groups. At the WAM's direction, the Contractor will work with the CEC to schedule working group sessions. The Contractor must demonstrate experience actively participating in meetings as a technical lead or expert, including supporting meeting facilitation, agenda development, and presentation materials.

Under the direction of the WAM, the Contractor shall coordinate and facilitate technical tasks to maintain alignment on scope, methodology, schedule, and deliverables.

The work in this subtask is expected to include, but not be limited to, the following:

Participate in reoccurring technical coordination meetings, held at a frequency determined by the WAM (anticipated to be biweekly unless otherwise directed).

Prepare and provide technical meeting agendas, presentation materials, and progress summaries.

Document key decisions, issues, risks, and action items in meeting notes.

Ensure attendance by the appropriate technical staff who can address data, modeling, methodology, and deliverable content.

Participate in special meetings, workshops, and events.

Maintain documentation of agreed-upon technical conventions, assumptions, and definitions used across tasks.

Work in this subtask may also include:

Supporting and scheduling ad hoc technical coordination meetings as requested by the CEC through WAs

Contractor Subtask Deliverables:

2.1.1 - Technical Meeting Agendas, Summaries, and Action Items Tracking.

2.1.2 - Presentation Materials (if requested).

Other deliverables to be defined as needed through WAs.

Subtask 2.2: Technical Review Cycles and Oversight

The goal of this subtask is to manage technical dependencies among tasks to ensure that all major technical deliverables are developed through structured, iterative review processes that support technical accuracy, clarity, and responsiveness to CEC feedback.

Under the direction of the WAM, the Contractor shall implement and manage technical review cycles for deliverables produced under this agreement.

The work in this subtask is expected to include, but not be limited to, the following:

Develop and follow multiple rounds of technical review, including: (1) internal Contractor review, (2) Subcontractor review (as applicable), (3) CEC initial review at thirty percent (30%), (4) CEC review at sixty percent (60%), and (5) CEC final review at ninety five or one hundred percent (95% or 100%) unless otherwise stated.

Incorporate CEC comments and provide revised drafts within timelines agreed upon with the WAM.

Maintain a comment log for significant changes, revisions, or resolutions to developed plan.

Ensure that all deliverables submitted to the CEC have undergone internal quality assurance and technical verification prior to submittal.

Monitor execution of technical tasks and flag technical risks or sequencing issues that may affect data usability, analytical integrity, or interpretation of results, and communicate such issues to the WAM in a timely manner.

Support identification of corrective actions when technical dependency issues arise, subject to direction from the WAM.

Work in this subtask may also include:

Adjusting review cycles or documentation requirements as directed by the CEC through WAs.

Contractor Subtask Deliverables:

To be defined as needed through WAs.

Subtask 2.3: Requirements Oversight

The goal of this subtask is to maintain clarity and alignment of research questions, analytical objectives, and information needs throughout execution, recognizing that research activities may evolve while ensuring that such evolution does not result in uncoordinated scope creep or analytical drift. Another goal is to ensure research activities remain consistent with applicable programming, funding, and external requirements, including those associated with federal requirements (see Standard Agreement Example, Attachment 6) and other applicable mandates.

Under the direction of the WAM, the Contractor shall monitor and manage requirements across all technical activities performed under this agreement.

The work in this subtask is expected to include, but not be limited to, the following:

Support on classification and documentation of core research questions, analytical objectives, and CEC information needs as technical work progresses across Tasks 2 through 11.

Document technical progress, data deliverables, and work progress for quarterly reports to DOE.

Maintain requirement tracking document, updated as needed, to ensure deliverables align with agreed-upon technical and analytical expectations.

Identify risks or gaps related to requirements and recommend mitigation strategies.

Ensure that all Subcontractors consistently adhere to established requirements and technical standards.

Coordinate with WAM to assess alignment of emerging technical directions with approved research intent, applicable requirements, and clarifications or adjustments within scope or require explicit direction.

Support structured change management for research-related requirements.

Work in this subtask may also include:

Supporting CEC responses to DOE or other oversight inquiries related to technical requirements, as directed.

Contractor Subtask Deliverables:

2.3.1 - Requirements Tracking Documentation (if requested).

2.3.2 - Technical Quarterly Reports (if requested).

Other deliverables as defined through WAs.

Subtask 2.4: Technical Coordination

The goal of this subtask is to ensure that all technical products such as spreadsheets, tools, dashboards, and datasets developed under this agreement are fully completed, documented, and transferred to the CEC in usable, accessible, and reviewable formats. The Technical Project Manager shall support the resolution and communication of technical issues requiring coordination across tasks or clarifications for CEC staff.

At the direction of the WAM, the Contractor shall support technical deliverable finalization and closeout activities.

The work in this subtask is expected to include, but not be limited to, the following:

Serve as a point of integration for addressing cross-cutting technical issues identified during execution.

Support the WAM by translating technical issues, constraints, or tradeoffs into clear, decision-ready technical information, as requested.

Coordinate responses to internal CEC technical feedback or external technical inquiries related to evaluation methods or analytical approach, without duplicating reporting or stakeholder engagement activities.

Ensure the completion of all technical deliverables for all technical tasks, following the agreed-upon review process.

Provide all underlying raw files, datasets, models, and tools used to generate the dashboard or other technical deliverables.

Provide all materials in non-proprietary and reusable formats, enabling CEC staff to independently review, replicate, and further analyze results.

Prepare a technical closeout package summarizing methodologies, data sources, assumptions, analytical workflows, and known limitations.

Ensure that all final deliverables comply with publication and accessibility requirements.

Maintain a secure, CEC-accessible data sharing portal that provides CEC staff with ongoing visibility into quantitative data collected under Tasks 5 and 6 as data is received and processed.

Update the data sharing portal on a regular and near-real-time basis, as data are collected and processed, without requiring completion of full analytical workflows.

Coordinate with the CEC to align portal structure, access permissions, and update cadence with project needs and WA direction.

Conduct periodic analysis readiness and alignment check-ins to confirm that ongoing field data collection remains consistent with approved evaluation design, analytical assumptions, and data requirements.

Manage technical staff and Subcontractors, as appropriate and as needed, to maintain risk registers, logs, or living documents used to communicate status and progress of project activities.

Work in this subtask may also include:

Addressing final technical clarifications identified during CEC acceptance or closeout, as directed.

Contractor Subtask Deliverables:

To be defined as needed through WAs.

Task 3: Evaluation Research Design

The goal of this subtask is to establish a foundational evaluation design for measuring and estimating compliance rates with the California Energy Code. This includes determining statewide compliance rates, breaking them down into full, process-based, and energy-based compliance, further analyzing compliance by phases such as plan review, installation, inspection, and other relevant groupings such as jurisdiction type, building category, and project type. This evaluation framework will guide subsequent tasks, including sampling and field data collection strategies. At the direction of the WAM, the Contractor shall develop a high-level evaluation design required before sampling (Task 4) and data collection (Tasks 5 and 6). The evaluation design should include these elements:

Subtask 3.1 Evaluation Framework - defines what compliance means, providing definitions, scoring logic, and phase structure.

Subtask 3.2 Measurement Model - defines how compliance is quantified, providing metrics, formulas, aggregation rules, and weighting logic.

Subtask 3.3 Research Design - defines how evidence will be generated, providing data requirements, sampling logic, and statistical methods.

Subtask 3.4 Implementation Tools - translates the research design into usable tools, providing draft instruments, checklists, forms, or structure to digest collected data to analysis.

Subtask 3.5 Independent Review - validates the design before execution, requiring Reviewer's approval and a quality assurance plan.

The evaluation design shall identify evaluation elements that are common across building categories, and which require category-specific approaches. This task requires iterative engagements from CEC staff and other subject matter experts as deemed appropriate by the CEC to finalize compliance rates.

Completion of Task 3 does not require finalization of sampling plans, execution of field data collection, or production of compliance estimates, which shall be addressed in subsequent tasks.

The work in this task is expected to include, but not be limited to, the following:

Subtask 3.1: Evaluation Framework

The goal of this subtask is to create an evaluation framework for evaluating compliance types with the California Energy Code across all major project phases, enabling consistent and measurable assessment of full compliance.

Under the direction of the WAM, the Contractor shall develop an evaluation framework, defining compliance components, scoring logic, and groundwork to guide or justify the compliance metrics.

The work in this subtask is expected to include, but not be limited to, the following:

Clearly define process-based compliance, energy-based compliance, and full compliance.

Describe how different compliance definitions may relate to one another and how any overlap can be interpreted.

Establish applicability and scope rules relating to the Energy Code.

Define the units of analysis (such as measure-level, system-level, project-level, and jurisdiction-level), and project phases (such as design, plan check, construction, inspection, and acceptance testing), and any other distinctions (such as climate regions, geographical considerations, and jurisdiction sizes).

Identify evaluation elements that are common across building categories and elements that require category-specific treatment.

Identify acceptable categories of documentation and evidence (such as compliance forms, energy models, plans, and inspection records) that can be collected under each compliance evaluation and what role they play in verifying compliance.

Compare and contrast two (2) to five (5) evaluation approaches per compliance definition (such as process-based, energy-based, and full compliance) to support CEC in selecting an implementation pathway.

Describe the proposed energy modeling approach(es), including applicable energy modeling software tools (such as CBECC, EnergyPlus, or other software), and any batching or aggregation methods to be used to support energy-based compliance assessment. This description shall address baseline and partial compliance scenarios, identify high impactful measures, and support technically defensible energy performance estimates.

Review critically relevant methods, tools, and literatures, including, but not limited to:

DOE Energy Code compliance evaluation protocols, including the use of energy modeling approaches such as representative "pseudo-building" or Monte Carlo-based simulations.

CEC modeling tools and methods for evaluating energy use, referencing CBECC and Alternative Calculation Methods documentations.

Energy Code Ace tools and documentation.

CPUC Evaluation, Measurement, and Verification (EM&V) studies that have used modeling for establishing related compliance values.

Provide a comparative assessment of the technical strengths, limitations, data requirements, and suitability of comparable alternative approaches relative to energy-based compliance evaluation methods and in addressing variation of accessibility for individual measures or projects in different construction phases.

Conceptual framework development to define unpermitted construction within the context of Energy Code compliance, including its relationship to process-based and energy-based compliance outcomes.

Identification of non-traditional data sources and signals that may be used, on an exploratory basis, to characterize unpermitted activity patterns (e.g., complaint-based information, enforcement records, market activity indicators, or other proxy datasets), subject to data availability and approval by the WAM.

Evaluation design options for incorporating unpermitted and unlicensed building construction projects into compliance analysis, including bounding approaches, qualitative characterization, scenario analysis, or sensitivity testing, as appropriate. Special attention is expected to be needed regarding recruitment of buildings for field testing and evaluation and avoidance of selection bias by failing to include all portions of the market that are doing projects without permits or licenses, wishing for this to go undetected (see the Background Summary in Section I: Introduction).

Documentation of assumptions, uncertainties, and methodological constraints associated with any proposed approach to ensure transparency and appropriate interpretation of findings.

Work in this subtask may also include:

Engage in iterative development of evaluation elements through coordination with the Reviewer (see Subtask 3.5), CEC staff and, where directed by the WAM, other interested parties, to inform and refine the design phase. Iterative coordination may include technical working sessions, targeted reviews, and incorporation of written feedback. Engagement coordination is for validation purposes only and will not change the approved evaluation methodology without prior CEC approval.

Contractor Subtask Deliverables:

3.1.1 - Draft(s) of Evaluation Framework Report.

3.1.2 - Final Evaluation Framework Report.

Other deliverables are to be defined as needed through WAs.

Subtask 3.2: Measurement Models

The goal of this subtask is to define how compliance will be quantified and aggregated once data is collected.

Under the direction of the WAM, the Contractor shall develop a measurement and aggregation model.

The work in this subtask is expected to include, but not be limited to, the following:

Specify how process-based, energy-based, and full compliance will be accessed on a binary (i.e., yes/no), graded (on a scale or by percentage), or hybrid basis (a combination thereof), subject to CEC's review. For example, one (1) of the possible approaches to process-based compliance is to assign weights to the compliance for each stage of permitting to reflect the relative impact on overall process-based compliance rate.

Define how partial compliance, non-compliance, probability of compliance, and unverifiable conditions are treated.

Define how compliance results are aggregated from measures, systems, or projects, based on the unit of analysis from Subtask 3.1.

Develop an equation for calculating full compliance rates, using a combination of, or selected data collected from process-based compliance and energy-based compliance established evaluation framework.

Define weighting or combination rules, if applicable, across compliance components with documented rationale, constraints, and CEC approval requirements.

Develop analytical approaches to quantify energy and cost impacts of non-compliance as defined and direct by the WAM, including structured modeling scenarios that simulate full compliance scenario (baseline) vs. partial compliance scenario (missing or incorrect measures).

Develop sensitivity analyses (change one (1) variable at a time - such as insulation level or HVAC efficiency - to see which measure compliance factors have the biggest impact) as appropriate.

Identify any testing instruments for measures or compliance requirements that may correspond to high-impact measures that are necessary to inform compliance levels or computation. Any proposed testing or verification of Energy Code measures shall be consistent with the applicable Energy Code provisions and shall be subject to CEC review and approval.

Once the approach for calculating compliance components is defined, describe the specific data elements or data evidence that will be required for data collection, validation, and management.

Define how unpermitted construction is addressed, including recruitment and selection bias issues resulting from market actors not wishing to disclose this work, including distinctions between process-based and energy-based compliance and identification of observable, partially observable, and unobservable dimensions.

Document assumptions, analytical boundaries, and any proxy indicators used to contextualize unpermitted activity.

Work in this subtask may also include:

Engage in iterative development of evaluation elements through coordination with the Reviewer (see Subtask 3.5), CEC staff and, where directed by the WAM, with other interested parties, to inform and refine the design phase. Iterative coordination may include technical working sessions, targeted reviews, and incorporation of written feedback. Engagement coordination is for validation purposes only and will not change the approved evaluation methodology without prior CEC approval.

Contractor Subtask Deliverables:

3.2.1 - Draft(s) of Measurement Model Report.

3.2.2 - Final Measurement Model Report.

Other deliverables to be defined as needed through WAs.

Subtask 3.3: Research Methodology

The goal of this subtask is to create a clear and scientifically sound approach for how to carry out the compliance evaluation. This means defining the methodology for generating the evidence that supports both the approved evaluation framework and measurement models.

Under the direction of the WAM, the Contractor shall develop a structured research methodology, clearly identifying the experiment design approach and statistical methods.

The work in this subtask is expected to include, but not be limited to, the following:

Identify key variable and analytical considerations derived from previous subtasks.

Define the research questions to be answered through qualitative and quantitative methods.

Define qualitative and quantitative methods for gathering and analyzing data, including statistical techniques, and building energy modeling for estimating compliance rates, uncertainty analysis, sensitivity testing, and/or others.

Describe the qualitative methods that may be used to interpret and explain quantitative compliance findings from field studies, such as identifying implementation barriers, process gaps, or behavior trends underlying observed compliance rates outcomes. The response shall describe how these qualitative methods complement quantitative data and how they would be prioritized based on relevance.

Create mock-ups of final products and an analysis plan for implementing the evaluation, including data collection workflows, and analysis steps.

Define modeling protocols for energy-based compliance and noncompliance evaluation, including selections of tools (such as CBECC, CBECC-Res, Energy Plus, or other modeling tools), input data requirements, and approaches for determining which inputs are derived from field data vs. building prototypes (such as schedules, occupancy, or other inputs), as well as scenario development for baseline vs. partial compliance cases.

Specify required input data and modeling assumptions.

Integrate energy modeling outputs into compliance scoring and cost/energy impact analysis.

Establish data collection protocols for ensuring data validity, reliability, and reproducibility for all compliance components and rates.

Establish defensible methodological approaches for addressing unpermitted construction and document sources of uncertainty, methodological limitations, and interpretive guardrails to ensure appropriate use and interpretation of findings.

Work in this subtask may also include:

Document any key assumptions and limitations encountered during research methodology development. This ensures transparency and makes the process easier to replicate.

Engage in iterative development of evaluation elements through coordination with the Reviewer (see Subtask 3.5), CEC staff and, where directed by the WAM, other interested parties, to inform and refine the design phase. Iterative coordination may include technical working sessions, targeted reviews, and incorporation of written feedback. Engagement coordination is for validation purposes only and will not change the approved evaluation methodology without prior CEC approval.

Contractor Subtask Deliverables:

3.3.1 - Draft(s) of Research Methodology Report.

3.3.2 - Final Research Methodology Report.

Other deliverables to be defined as needed through WAs.

Subtask 3.4: Evaluation Support Tools and Documentation

The goal of this subtask is to provide practical and implementation tools that support consistent and efficient execution of the approved evaluation design.

Under the direction of the WAM, the Contractor shall develop tools and documentation to support implementation of the evaluation design.

The work in this subtask is expected to include, but not be limited to, the following:

Create a list of required tools to support evaluation implementation, subject to CEC review and approval.

Develop templates, checklists, and tools to support compliance component evaluation activities across all stages, aligned with Subtasks 3.1 to 3.3.

Draft guidance documents using plain language and visual elements, where appropriate, to clearly demonstrate how the evaluation framework and measurement models are applied in practice, including, but not limited to, illustrative examples, workflows, or diagrams intended to support consistent implementation.

Coordinate with CEC staff and subject matter experts deemed appropriate by the CEC to refine evaluation criteria and ensure technical accuracy.

Establish data collection protocols that consider stakeholder relationships and desired levels of engagement from points of contact. Ensure that data protection and handling practices are applied to address confidentiality, privacy, or other potential concerns. Consider how data should be collected and the format (such as field photography, compliance forms, other data types) of the data for it to be a viable source for compliance assessment or back-up data, including how any data types might be used and where it might prove too burdensome/costly for the targeted sampling population.

Establish tools to support ingestion and management of site-level data records, allowing multiple observations or data sources per site to be linked within a structured dataset and enabling calculation of key metrics from the underlying data. Provide a mock-up or relationship diagram, illustrating the proposed database structure, data dictionary, data sources, coding and editing procedures, and other important information for potential users of the data, subject to CEC's review and review of other interested parties deemed appropriate by the CEC.

All tools developed under this subtask are provisional and shall not be used in field data collection until reviewed and approved following completion of Subtask 3.5 work and deliverables.

Work in this subtask may also include:

Engage in iterative development of evaluation elements through coordination with the Reviewer (see Subtask 3.5), CEC staff and, where directed by the WAM, other interested parties, to inform and refine the design phase. Iterative coordination may include technical working sessions, targeted reviews, and incorporation of written feedback. Engagement coordination is for validation purposes only and will not change the approved evaluation methodology without prior CEC approval.

Contractor Subtask Deliverables:

3.4.1 - List of Evaluation Implementation Supporting Tools and Documents.

3.4.2 - Evaluation Implementation Support Tools.

3.4.3 - Evaluation Implementation Guidance Documents.

Other deliverables to be defined as needed through WAs.

Subtask 3.5: Statistical Design Review and Quality Assurance

The goal of this subtask is to ensure evaluation design is methodologically sound, fit for purpose, and defensible given budget and implementation constraints. This includes an independent qualified technical and statistical review prior to the data collection to verify that the evaluation framework, measurement models, and research methodology that outline analytical assumptions and data quality controls are appropriate and consistent with accepted evaluation best practices.

Description of the Qualified Statistician Reviewer (Reviewer):

The Contractor shall ensure at least one (1) Qualified Subcontractor Firm that will be dedicated to Qualified Statistician Review is included as a part of the Project Team. The Qualified Statistician Reviewer (Reviewer) employed by this Subcontractor Firm must be an independent qualified technical advisor with appropriate academic training and a minimum of five (5) years of professional experience in statistics, biostatistics, econometrics, applied mathematics, or closely related field. The Reviewer shall be independent of study implementation and shall not perform field data collection or analysis activities under this agreement.

The Reviewer is expected to provide objective, independent review of the proposed study design and analytical assumptions in the field data collection methods, with the purpose of minimizing methodological risk and supporting the technical credibility and defensibility of study outcomes.

Prior to commencement of field data collection, the Reviewer shall review and validate, at a minimum:

The proposed evaluation and research framework;

The sampling strategy and power assumptions;

Any proposed randomization or comparison group methodology, if applicable; and

The planned analytical approach prior to commencement of field data collection.

The Reviewer shall have advisory authority with respect to the study design, sampling strategy, power assumptions, and planned analytical approach. The Reviewer responsibility is limited to technical review, advisory input, data quality assurance, and analysis quality assurance. The Contractor retains the full responsibility for proposing and implementing the study design, subject to CEC approval. The Reviewer does not replace the Contractor's responsibility for delivering the work, nor does the Reviewer exercise managerial or contractual authority over the Contractor.

All deliverables completed by the Reviewer must be submitted directly to the CEC. The Reviewer must report directly to the CEC on key design decisions and disclose any actual or potential conflict of interest. The CEC reserves the right to approve, reject, or replace the designated Reviewer with another Reviewer from another Qualified Subcontractor Firm.

The Contractor shall cooperate with the Reviewer by providing requested design documentation and draft deliverables in a timely manner and by engaging in good-faith technical discussions to address issues raised by the Reviewer. In the event of methodological disagreements between the Contractor and the Reviewer, the Reviewer shall document the issue, the basis of concern, and any recommended alternatives in writing. Final decisions regarding the study design shall remain with the CEC, informed by the Reviewer's input.

In addition to study design review, the Reviewer may be engaged during and following field data collection to support ongoing quality assurance and validation of data collection and analytical practices. Such quality assurance activities may include review of field protocols, limited data quality checks, or post-collection validation. The scope, timing, and level of effort shall be defined in subsequent tasks and WAs and shall remain subject to CEC approval.

The CEC must review and provide written approval for these deliverables before any subsequent steps proceed.

Contractor Responsibilities:

Under the direction of the WAM, the Contractor shall engage and coordinate with the Qualified Statistician Reviewer to provide technical review and quality assurance prior to and after the field data collection. The Contractor shall identify in their proposal all Subcontractor Firms who possess the expertise required to complete Qualified Statistician Review. The Contractor must establish and maintain organizational separation of duties between the Subcontractor/employees providing Qualified Reviewer services from other employees/Subcontractors providing services under this agreement. In the event the Contractor is uncertain whether a separation of duties is required, the Contractor shall submit a written description of the relevant details and shall avoid assigning individuals to perform the activity in question until receiving written approval from the WAM.

The work in this subtask is expected to include, but not be limited to, the following:

Identify, propose, and engage a qualified independent Qualified Statistician Reviewer for CEC approval, including submission of qualifications, resume, supporting certifications, examples of prior work, and disclosure of any conflicts-of-interest.

Provide requested study design documentation and draft deliverables to the Reviewer in a timely manner.

Ensure that the Reviewer deliverables are incorporated into the project's schedule and deliverable progress is tracked accordingly.

Develop a Data Quality Assurance Plan to ensure accuracy, consistency, and integrity of data throughout the study. The Data Quality Assurance Plan shall describe the process for ensuring data quality and integrity of data collected during field and administrative data collection activities. The Data Quality Assurance Plan shall also describe the process for ensuring validity, transparency, and reproducibility of analytical methods and assumptions applied to the data. The Data Quality Assurance Plan shall include, at a minimum:

Identification of roles and responsibilities in the quality assurance process. For example, the quality assurance process shall be performed by the Contractor, with independent validation by the Reviewer.

Documentation of protocols for data quality validation, handling, storage, and security.

Documentation of protocols for maintaining validity, consistency, transparency, and reproducibility of analytical methods. For example, describe procedure for handling revisions to data collection, tools, or methodologies.

Identification of potential risks associated with data collection and analysis, and corresponding mitigation strategies.

Ensure the quality assurance plan explicitly references the templates, checklists, guidance developed in Subtask 3.4

Coordinate with the Reviewer on developing quality assurance protocols and procedures.

Create an evaluation design risk register having identified potential risks in data collection and analysis, with assessment of likelihood and impact, and proposed mitigation strategies to minimize disruptions or biases.

Engage in good-faith technical discussion to address issues raised by the Reviewer.

Incorporate CEC-approved revisions resulting from the Reviewer's input prior to the initiation of field data collection.

Contractor Subtask Deliverables:

3.5.1 - Qualified Reviewer Qualification Package.

3.5.2 - Data Quality Assurance Plan Report.

3.5.3 - Evaluation Design Risk Register.

Other deliverables to be defined as needed through WAs.

Reviewer Subtask Deliverables (Submit Directly to the CEC):

3.5.4 - Concurrence Memorandum: Evaluation Design that confirms the proposed design (Subtasks 3.1 to 3.3) meets accepted methodological standards. Explain assumptions and limitations. Must explicitly state whether Subtask 3.4 tools and guidance are approved for field use or require further revision.

Task 4: Sampling Plan Development and Validation

The goal of this task is to develop a feasible and defensible sampling plan that supports the CEC-approved evaluation design established in Task 3. The sampling plan shall be designed to support credible assessment of process-based compliance, energy-based compliance, and full compliance within defined schedule, cost, and access constraints, and shall not modify the approved evaluation framework without prior written approval from the CEC.

While the Task 3 evaluation framework may already identify broad categories and compliance components, the sampling plan in Task 4 will apply relevant data sources to identify targeted samples through applying eligibility checks and sampling rules. This task is expected to involve iterative development through structured engagement with the CEC to balance methodological rigor, cost, and feasibility.

Task 4 is structured as follows:

Subtask 4.1 Sampling Framework - explores and documents the possible sampling options, data sources, grouping approaches, and risk considerations, with structured iteration with the CEC to understand tradeoffs.

Subtask 4.2 Sampling Plan - turns those options into a practical, cost-aware sampling plan that explains how selection, tracking, and replacement would actually work.

Subtask 4.3 Validation - serves as a formal decision gate, where the sampling framework and plan are independently reviewed and then approved by the Reviewer and CEC, locking in the final design before fieldwork begins.

Subtask 4.4 Risk Assessment & Mitigation - focuses on monitoring and managing risks during implementation using pre-approved mitigation actions, without reopening design decisions.

Subtask 4.5 Sample Recruitment Support - carries out the approved plan by recruiting jurisdictions, projects, and sites, and tracking participation.

Subtask 4.6 Sampling Implementation Guidance - documents clear, procedural guidance so that everyone executes the approved sampling plan consistently.

At the direction of the WAM, the Contractor shall develop a comprehensive sampling plan to address the applicable study populations and data sources, including jurisdictions, project records, and sites or buildings where verification activities may occur. All sampling decisions shall be explicitly documented, justified, and subject to the CEC review and approval prior to implementation.

The work in this task is expected to include, but not be limited to, the following:

Subtask 4.1: Sampling Framework

The goal of this subtask is to define sampling options and assumptions that will inform downstream planning.

Under the direction of the WAM, the Contractor shall identify potential study populations, data sources, grouping approaches, inclusion and exclusion criteria, known limitations/risks, and sample size assumptions. This subtask shall involve iterative engagement with CEC to evaluate tradeoffs among methodological rigor, cost, and feasibility. The output of this subtask establishes the baseline sampling assumptions and risks that will guide subsequent tasks.

The work in this subtask is expected to include, but not be limited to, the following:

Identify and describe study populations for each compliance component consistent with Task 3 definitions and building categories.

Specify all possible sampling data sources for each compliance component, such as:

Process-based frame (such as permit applications, enforcement records, or plan review, inspection logs); and

Energy-based frame (such as energy models, post-occupancy datasets, or commissioning reports).

Address how energy-based compliance data may be collected from sampled buildings, including strategies for aggregating inputs and applying batch energy modeling to optimize analytical efficiency while minimizing cost and participant burden.

Identify and compare alternative grouping approaches to support representative sampling and target subgroup analysis. Grouping factors may include, but not limited to, differences in jurisdiction capacity (such as staffing, experience, and historical compliance rates), geographic location, CEC climate zones, project type (such as new construction vs. existing buildings/alterations), building type (such as office, retail, school, multifamily, industrial/warehouse, healthcare, or others), permit valuation (small, medium, or large), compliance method (prescriptive vs. performance), and compliance measures (such as HVAC, envelope, lighting, controls, and other applicable measures defined in Task 3).

Assess tradeoffs among alternative grouping approaches, including impacts on statistical precision, operational complexity, sample size requirement, and cost.

Define inclusion and exclusion criteria like minimum documentation completeness, eligibility for data access, feasibility for site visits, safety considerations, and measure-specific eligibility for energy-based compliance evaluation, as applicable.

Determine preliminary sample size calculation and allocation logic such as setting precision targets (such as margin of error and confidence level) and power assumptions for detecting meaningful differences across strata or compliance components to support feasibility and cost discussions.

Identify assumptions, limitations, and known sources of bias, including expected non-response, data access constraints, correlation structures, and clustering effects.

Synthesize and recommend a grouping approach and data source for CEC consideration, clearly documenting rationale for the recommendation and tradeoffs associated with alternative approaches.

Define and compare a range of sampling options for characterizing unpermitted construction, as feasible.

Document assumptions, data requirements, feasibility, sampling boundaries, inclusion criteria, and known sources of coverage bias associated with each sampling option.

Work in this subtask may also include:

Engage in iterative development of sampling framework elements through coordination with the Reviewer (see Subtask 3.5), CEC staff and, where directed by the WAM, other interested parties, to inform and refine the design phase. Iterative coordination may include technical working sessions, targeted reviews, and incorporation of written feedback. Engagement coordination with other interested parties is for validation purposes only and will not change the approved evaluation methodology without prior CEC approval.

Contractor Subtask Deliverables:

4.1.1 - Draft(s) of Sampling Framework Report.

4.1.2 - Final Sampling Framework Report.

Other deliverables to be defined as needed through WAs.

Reviewer Subtask Deliverables (Submit Directly to the CEC):

4.1.3 - Concurrence Memorandum: Sampling Framework (provides independent advisory input on the alternative sampling approaches considered, the recommended sampling framework, and associated power and feasibility assumptions).

Subtask 4.2: Sampling Plan

The goal of this subtask is to translate the approved sampling framework into a practical and comprehensive sampling plan for sampling selection, tracking, and replacement.

Under the direction of the WAM, the Contractor shall develop a detailed sampling plan which specifies how the approved sampling framework will be implemented in practice. The sampling plan developed under this subtask may be refined iteratively in coordination with the CEC to address feasibility and cost considerations identified during planning, provided refinements remain consistent with the approved sampling framework and do not materially alter stratification logic or sample size targets without prior written CEC approval.