|Agency:||State Government of Pennsylvania|
|Type of Government:||State & Local|
|Posted Date:||Aug 16, 2019|
|Due Date:||Aug 30, 2019|
|Bid Source:||Please Login to View Page|
|Contact information:||Please Login to View Page|
|Bid Documents:||Please Login to View Page|
Department of Education
Request for Application:
Computer Science 7-12 Educator Certification Test
The Commonwealth of Pennsylvania, Department of Education () is interested in receiving Requests for Application (RFA) from companies that develop, administer, and score educator certification tests for the newly approved Computer Science 7-12 certificate. Qualified companies shall demonstrate the capability of providing testing to candidates that aligns with Pennsylvania standards, of providing exemplary customer service, as well as producing accurate, timely, and comprehensive score reports. While not required, experience with providing educator certification testing in other states is desirable.
The purpose of ’s request is to determine the certification testing options available and the level of responsiveness to satisfy the requirements set forth herein. will enter into one or more agreements with vendor(s) to provide these assessments and services to make them available to educator candidates in the Commonwealth, as well as to ensure tests remain robust and can meet changing requirements.
PDE approved its new Computer Science 7-12 certificate guidelines in December 2018.The subject specific standards for the competencies within the program are based upon the Computer Science Teachers Association (CSTA) Revised Standards. The CSTA standards were endorsed by the Pennsylvania State Board of Education.
PDE requires testing for all certificate holders on their subject matter content prior to receiving certification and is now in the position of identifying a certification test that covers the competencies by which the certificate is built upon. This RFA has been written for this need.
PDE will enter into agreements with testing vendors that can provide exemplary customer service; clearly align test content to Pennsylvania’s Core Standards and Academic Standards; provide accurate, timely, and comprehensive feedback to all entities; and outline clear accountability measures as described within this document.
seeks RFAs from companies that will provide an assessment for candidates for educator certification in computer science 7-12. In this RFA, PDE sets the stage for some key guiding principles regarding educator assessments, including:
1. Exemplary Customer Service: expects that all key stakeholders, including test takers, educator preparation programs, and , will receive exemplary customer service. Woven throughout this document is a strong emphasis on an overall quality experience, from customer service hotlines to score reports to test administration.
2. Clear Alignment with Pennsylvania’s Core Standards and Academic Standards: Given the specificity of Pennsylvania’s Core Standards and Academic Standards, there is sufficient guidance regarding the competencies that candidates are expected to acquire. The assessments are expected to closely align to these standards to ensure that candidates are tested only on Pennsylvania-based competencies.
3. Accurate, Timely, and Comprehensive Feedback to All Entities: This high stake test provides an important feedback loop for all key stakeholders: candidates await results to obtain certification; Pennsylvania awaits results to inform statewide policy and report to the federal government; and educator preparation programs await results to make decisions about program direction and candidate supports. It is, therefore, crucial that testing vendors provide accurate, timely, and comprehensive feedback to all entities.
4. Clear Accountability Measures: As one of the largest producers of teachers in America, Pennsylvania should be a model state in all aspects of certification, including testing. Test development, administration, security, and other components detailed in this document are designed to ensure that vendors are held accountable for all elements of the testing process. Annual Technical Advisory Committee meetings must be held to review test performance, test item statistics, and performance. Vendors must implement best practices as addressed in the Standards for Educational and Psychological Testing (rev. 2014).
5. PDE’s Participation in Test Development and Administration: The vendor will ensure PDE staff are actively engaged in all aspects of test development activities. PDE will have final authority and approval in test development activities and the final authority in resolutions and decision making.
6. PA Faculty and Educator Participation in Test Development, Maintenance, and Updates: The vendor will ensure that item review, data review, standard-setting, bias, sensitivity, and fairness review, and other such committees have representation from subject-matter experts working at PA universities and schools. Tests developed solely for PA must use PA subject-matter experts exclusively.
has issued certification program-specific guidelines that list compeencies for each of he above certification areas that address grade-specific mehods, pedagogy, and child developmen as well as conen in core subjec areas ha align wih Pennsylvania’s Core Standards and Academic Standards. The Computer Science 7-12 Guidelines are posted on PDE website.
encourages and values innovation and creativity in the vendor’s RFA. This document outlines its expectations for the tests and the contractual commitments it seeks to impose upon the company or companies whose tests are approved for the Commonwealth.
TIMELINE: All work for full educator certification test development, including appropriate field testing, must be performed so that all assessments are available for administration by September 1, 2021.
DUE DATES: Written RFAs must be received by August 30, 2019 at 3:00 PM Eastern Standard Time. Timing for evaluating responses and scheduling of presentations depends on the quantity and quality of responses received.
To be considered for this solicitation, the vendor must submit an electronic copy emailed to the PDE's Procurement Office. Email submissions should be sent to Mark Donegan, procurement manager PDE, who will serve as Point of Contact on this RFA at: firstname.lastname@example.org. If you choose to submit in hardcopy form, please mail the original to:
PA Department of Education
Bureau of Management Services
, 15th Floor
Attn: Mark Donegan, Procurement Officer
Once the Commonwealth has received the RFA, it will contact all respondents determined reasonably responsive and may require additional points of clarification at that time.
Responses should be succinct, preferably no more than 75 pages, but must include all information requested (A through K). Sample documents, forms or policies may be attached as a reference in addition to the 75-page Request For Application.
Content of Expression of Interest
Vendors are to submit succinct responses to each item listed below. Follow the number sequence used for each section (i.e., A, 1, a, b, c, 2, etc.)
A. Written Assurances from the Vendor
The vendor assures PDE that:
1. The content test is aligned with the appropriate Pennsylvania Core Standards and Academic Standards, as well as the program specific-guidelines.
2. The production of all testing materials is performed under the direct supervision of the vendor, and both manual and statistical quality control is exercised over all phases of materials production.
3. Secure processes and procedures will be implemented to maintain security of all assessments and data.
4. Computer-generated testing reports will be high quality, easy to read and understand, properly prepared, and accurate.
5. Electronic copies of test reports for candidates, PDE, and the Pennsylvania educator preparation programs will be available via a secure web site.
6. Individual score reports will be provided to candidates and aggregated score reports will be provided to PDE and Pennsylvania educator preparation programs within 30 days from the testing date.
7. Testing data will be available to without requiring manual updating.
8. Data files will be backed up daily.
9. It will conduct a successful backup/restore test at least once a year.
10. The assessment will align with the Program Framework Guidelines, Computer Science 7-12.
11. Reasonable accommodations and adaptations for all persons needing special accommodations because of a disability or other warranted need, alternate testing days because of a religious conflict, or modifications for other reasons.
B. Experience, Background, and Qualifications
1. Describe qualifications and experience providing certification tests for candidates. Include the following:
a. Number of years engaged in testing;
b. States where testing is currently provided;
c. Number of locations served within Pennsylvania, if applicable;
d. Number of locations served in bordering states; and
e. Identify the assessments currently offered to candidates for certification.
2. Provide examples of prior certification testing program experience, if any. List a minimum of three references, including contact name, title, and telephone or email contact information for individuals for whom you have performed similar testing services within the last three years. PDE will contact each reference listed.
3. Provide résumés of each content area leader for each certification area who will participate in face-to-face item review meetings with PDE staff and/or PA educators.
4. List all errors that occurred in the past two years that were the responsibility of the vendor or its subcontractors. An error is defined as any event that results in any incorrect/incomplete results being reported, printing errors, or any delay in meeting the scheduled delivery of any component of the assessment program. Include the following information for each error:
a. Scope of the error (i.e. number of candidates, administration site, etc.);
b. Identification of who identified the error (e.g. vendor, client, candidate, etc.);
c. How errors were resolved;
d. Systemic changes implemented to prevent recurrence of the same or similar problems; and
e. Name, address, and phone number of the client contact person who can respond to inquiry regarding the impact of the error on the assessment program.
C. Project Management Plan
1. List all certification areas vendor is interested in implementing and administering in Pennsylvania.
2. Provide a Project Management Plan that includes a work plan for each task. The detailed plan should outline all elements of the test development process from the anticipated date of the contract through September 1, 2026 with key deliverables and dates and benchmarks, including regular, on-going update meetings with PDE, content validation activities, standard setting studies, field testing, and test administration. Where appropriate, a PERT or GANTT chart display should be used to show project, task, and time relationship.
a. Describe the vendor’s capacity to develop, implement, and administer all proposed assessments by September 1, 2026.
b. Develop a two-year time line that identifies dates of testing, candidates’ registration deadlines, dates for issuing score reports, and dates when preparation entities will match candidates with score reports for federal reporting purposes. It is expected that vendors will provide results to all relevant parties as quickly as possible.
3. Describe how all Commonwealth requirements for Archive, Business Continuity, and Disaster Recovery will be met. Ensure that:
a. The assessment system, including data files and online applications, will be available to and its customers with a minimum of interruption or delay. For applications, the goal is 99% availability between 6:00 a.m. and seven days per week. After a declared disaster, recovery time must be no longer than two weeks. Any system architecture considerations or other additional resources to meet these requirements must be included in your RFA.
b. For archive and business continuity purposes, the vendor must:
i. Design a solution that has the ability and functionality to archive data that is six years old or for the life of the contract with PDE; the system must be able to support both online and offline archival and retrievals.
ii. Develop a recovery and back up procedures in all aspects of the system’s life cycle, including implementation and operational environments.
iii. Implement a solution that is architected to provide redundancy and fault tolerance. Capabilities such as clustering, load balancing, database replication, at a minimum, must all be used for the proposed solution. The Solution Provider must develop the system design and architecture to prevent any single point of failure from adversely affecting the assessment system.
c. For disaster recovery purposes for this project, the vendor will:
i. Develop and document a Disaster Recovery plan for the certification assessment system. The scope of the recovery plan should be a total restoration of the hardware/software, applications, databases, and all relevant technology components and platforms to support the assessment system.
ii. Plan for and implement a formal test of the backup/restore processes. The test must be executed and be successful in restoration of all services, components, and application without failure. The plan must list criteria for determining success of backup/restore and how failures will be documented and submitted to PDE. Minor or minimal impact issues could be items such as documentation updates, facilities, environment, or delays.
4. Submit an “as required” report, identifying problem areas. The report should describe the problem and its impact on the overall project and on each affected task. It should list choices with advantages and disadvantages of each and include recommendations with supporting rationale.
D. Test Development
Whenever possible, vendors should create assessments using a modular format. In this way candidates can re-take only the module where his/her score was below the cut score.
1. Describe how the Computer Science 7-12 assessment will address the accommodations and adaptations for diverse learners in inclusive settings and English language learners’ competencies.
2. Describe the process that will be used by the vendor in conducting standard setting sessions for each certificate area and how PDE staff will be involved.
3. Develop a plan for scoring modules in each assessment so that candidates who do not pass the certificate area assessment may retake only the module(s) in which they were not successful.
4. Describe how the vendor will establish composite scoring, where applicable.
5. Provide a minimum of five sample test items (including passages, graphics, and scenarios as appropriate) for each certification assessment area to be implemented/administered in Pennsylvania.
a. Include the steps that will be taken to develop or provide items in sufficient quantity to cover five years of assessments.
b. Describe a design for maximum use of items.
6. Describe how vendor will recruit and form a Technical Advisory Committee () and propose a meeting schedule for each certification testing area to be discussed.
7. Submit at least one sample Technical Report for one assessment area. The final scope of the Technical Report, which will be submitted annually to PDE, will be negotiated with the TAC but, at a minimum, will include analyses and information related to:
a. The assessment development procedures, including assessment blueprints and other documentation.
b. A report of the item analysis results including a complete set of item statistics.
c. An analysis of scorer consistency and scorer drift for the open-ended response items on the assessments.
d. A report of the Differential Item Functioning (DIF) analyses as well as any subsequent actions taken because of these analyses.
e. A report of the assessment reliability and standard errors of measurement, including measures of decision consistency and decision accuracy regarding the performance level determinations of students.
f. A standards-setting report.
g. Evidence documenting the validity of the score inferences from those assessments. While it may be necessary for vendors to present content and/or predictive validity evidence, it will not be sufficient.
8. For the assessment area, propose multiple assessment designs using different combinations of selected-response and constructed-response items.
9. Outline the process for assessment construction, including the following:
a. A breach form policy;
b. The field testing plan, including bias, sensitivity, and fairness reviews;
c. Total number of forms needed to ensure a quality process; and
d. Assessment construction analyses necessary to support the construction of technically sound modules and forms.
10. Demonstrate responsibility for establishing and documenting evidence of the validity of the assessment scores and referencing the professional standards applied to establish validity. Specify all calibration and scaling procedures to be utilized.
11. Describe the procedures and statistical and psychometric analyses to be used to evaluate all items, including field test items and operational items (including modified and accommodated administrations). Analyses must include standard item analysis statistics, including p-values, point-biserials, and DIF analyses.
12. Propose a comparability study of the computer- and paper-based assessment modality as appropriate or provide reports of studies already conducted.
E. Test Administration and Scoring
1. Develop a plan for scanning/imaging and scoring selected response items that addresses each of the following:
a. Quality assurances, including:
i. Document and data processing;
ii. Reconciliation plans;
iii. A security plan for handling assessment materials upon arrival at the processing center;
iv. Logistics of preparing documents for scanning;
v. Handling of documents that appear impossible to scan;
vi. The number of assessment materials reconciliations;
vii. Matching constructed responses and multiple-choice questions to candidate information;
viii. Batch determination;
ix. Images scanned for hand scoring;
x. Conducting pre-editing to identify suspected errors and omissions for the vendor’s editing staff’s checking and disposition; and
xi. Conducting post-editing to make certain that all data is correct and all corrections are valid.
b. Systems and equipment to scan and transcribe all data from the answer documents onto an electronic data file that is a single, individual record; and Standards setting for all assessments, including a cut score recommendation to PDE for each assessment and module.
2. Recommend and implement a plan for administering the tests, ensuring access and availability across the Commonwealth. The proposal shall contain a comprehensive and detailed plan for all aspects of test administration, including but not limited to:
a. Examinee and client support and services;
b. Information dissemination;
c. Secure materials production and shipping;
d. Selection of the site staff (including procedures to preclude illegal discrimination in the selection of site staff), staff training and use (e.g., number of proctors per examinee), testing of individuals needing modifications, and site management including on-site test security and problem resolution;
e. Examinee registration and on-site identity assurance and documentation;
f. Test site characteristics such as proposed specifications and geographical locations;
g. Scoring processes (including holistic and analytic scoring, and reporting performance by domain and competency) and quality assurance;
h. Maintenance of historical and future test records, examinee data, and documentation; and
i. The implementation of technology for accessibility, accuracy, and cost-effectiveness.
3. Prepare a draft of a Pennsylvania-specific Annual Bulletin that includes test administration dates, qualifying scores for each assessment and module, procedures for requesting modifications/accommodations, timeline for cancellation of tests, etc.
4. List supplemental test materials for candidates vendor will develop or make available, including but not limited to:
a. General rubrics for open-ended items; and
b. Mathematical formula or other reference sheets.
5. Describe the vendor’s policies for scoring both constructed responses and open-ended items related to each of the following:
a. Range-finding sessions.
b. Scorer training using scoring guidelines and anchor sets reviewed in collaboration with various stakeholders.
c. Training procedures for scorers of open-ended items and the protocol and procedures to qualify scorers.
d. Monitoring and ensuring reader accuracy and reliability so that scoring is fair and consistent across readers and assessment administrations.
e. Protocols to ensure consistency in the work of scorers. The protocol will include the processes for daily checking of reader accuracy, an audit of 10% of each reader’s scores by a lead reader, and the rescoring of responses scored by readers identified as failing to meet standards for accuracy.
f. Procedures to ensure consistency in the work of scorers across years.
g. When the vendor will submit to PDE summary reports of open-ended scoring sessions.
h. How the vendor will conduct annual scorer drift studies both for internal consistency and consistency across years.
6. Include a previously developed documented report of the open-ended scoring process; the report should be no more than two years old.
7. Describe how the vendor will verify the accuracy of all scanned data and documentation procedures for any irregularities.
8. Describe how the vendor will administer tests to military personnel, including non-military individuals employed at Department of Defense educational facilities, and outside Pennsylvania for individuals who seek Pennsylvania educator certification. A single fee per test shall be proposed for administering the tests outside of within bordering states. The proposal must include a recommended list of cities within each state, subject to approval, the test can be administered to examinees either on-demand within six weeks of receiving a request from an examinee or at reasonably scheduled periodic intervals two or three times annually. The fee proposed shall be sufficient for the vendor to recover costs of this test administration.
9. Describe the procedures that will be implemented to maintain the security of assessment materials throughout the assessment administration process.
10. Describe how all materials, both electronic and hard-copy, are stored in an orderly fashion so they are quickly retrievable to respond to requests. In the event materials have been inaccurately processed, the vendor will reprocess them without additional cost. Vendors must be responsible for the timely and accurate collection of all assessment materials from testing sites to meet the established guidelines. All documents with candidate names will be stored in a secure environment accessible only to authorized personnel.
11. Describe how 100 percent accuracy in all scoring operations will be demonstrated by the vendor.
12. Specify all equating procedures that will be used to ensure comparability of scores across forms and years being certain to include issues of positioning of items and their impact on equating. requires open-ended items be included in the equating.
13. Describe the procedures that will be utilized to ensure comparability of scores from one assessment administration to the next administration or provide copies of studies already conducted.
F. Equity Assurance
1. Describe how the vendor will assure and implement reasonable accommodations and adaptations for all persons needing special accommodations because of a disability or other warranted need, alternate testing days because of a religious conflict, or modifications for other reasons. Large-print and Braille versions of each assessment and/or audio recorded versions for the visually impaired will be provided by vendors.
2. Describe the vendor’s commitment to ensuring equity and access in all aspects of assessment. All activities shall comply with or exceed appropriate ethical and legal guidelines, including but not limited to the American with Disabilities Act (ADA) and American Psychological Association () standards.
3. Describe how the vendor will form and train an Equity Assurance Panel (EAP) to review test frameworks, all test items, and other materials and policies as appropriate. PDE will assist in identifying panel members.
H. Support of Stakeholders
1. Provide supports for candidates, Pennsylvania educator preparation programs, and , including but not limited to the following supports:
a. Candidate Supports:
i. 24-hour toll-free candidate hotline;
ii. Options for computer-based and paper-based testing;
iii. Representative sample test items available at no additional cost;
iv. One representative sample assessment in its entirety per certificate area available at no additional cost that candidates can take as a scored, practice assessment;
v. Detailed score reports, disaggregated by module and/or Major Competency Area;
vi. Diagnostic analysis for repeat test-takers;
vii. Provide multiple payment options; and
viii. A secure website to access information and register for tests.
b. Educator Preparation Program Supports:
i. One representative sample assessment in its entirety per certificate area available at no additional cost that faculty can take as a scored, practice assessment;
ii. Representative sample test items available at no additional cost;
iii. Faculty guides for each certificate area assessment to assist faculty members in preparing candidates; and
iv. Score report options.
c. PDE Supports:
i. Conduct customer satisfaction surveys and allow state to access feedback;
ii. Meet federal government reporting requirements;
iii. Score report options;
iv. Provide information/data in a format that ensures compatibility with data systems ( will provide compatibility specifications);
v. A secure website for to access data (include the posting of annual summary files by certificate area);
vi. All electronic data files (the individual candidate score reports) will be forwarded to at the end of each month. must approve all file formats;
vii. Collect candidate student demographic and program information from all test takers to meet the needs of PDE; and
viii. Provide statistics on national (if applicable) and performance, including mean and standard deviations for all test areas, unless statistical limitations apply, in which case an explanation is required.
2. Include samples of all test reports that will be prepared for candidates, PDE, and the Pennsylvania educator preparation programs.
3. Describe at least two ways and the costs associated with each method the vendor will make test reports available to candidates, PDE, and Pennsylvania educator preparation programs.
4. Describe the timeframe and costs for vendor to provide an expedited testing report.
5. Describe how candidates can obtain and cost to access to future score reports.
6. Include sample reports and data systems which allow institutions of higher education and to be able to access multiple years of testing data, starting with the tests administered in September 1, 2021.
7. Describe how the vendor will not create testing barriers for out-of-state educators looking to become certified in Pennsylvania by conducting comparability analyses of other state’s existing or new tests.
I. Financial Capability
Describe your firm’s financial stability and economic capability to perform the contract requirements. Audited financial statements or tax returns no more than two years old will be accepted in response to this item.
1. In a table format list cost estimates for all services disaggregated by options, including, but not limited to:
a. The total cost to the candidate for each assessment (the vendor may use historical test taking data available on ’s website at http://www.teaching.state.pa.us.) Vendors – not the candidates – will absorb the cost of any identified errors;
b. Candidate registration fee and when candidates would be charged a registration fee;
c. Costs for candidates to retake tests or modules;
d. Costs of all other services available to candidates, Pennsylvania educator preparation programs, and PDE; and
e. Anticipated increases that might occur during the next five years.
2. Describe how the vendor will provide a Pennsylvania candidate with the opportunity to retake a test without incurring additional costs and what criteria must be met for this to be applied. Previous criteria in the past included:
a. Candidate had a 3.0 GPA or higher (the Dean of the School of Education or the Superintendent of the employing local educational agency may waive the GPA or the score point requirement if they feel the individual warrants special consideration); and
b. Candidate missed the passing score by a minimal margin to be determined by the vendor (the Dean of the School of Education or the Superintendent of the employing local educational agency may waive the GPA or the score point requirement if they feel the individual warrants special consideration); and
c. Candidate has not taken the test more than two times.
K. ADDITIONAL REQUIREMENTS
Presentations: After a review of the submissions, the Commonwealth reserves the right to request presentations regarding the Expressions of Interest received from interested companies. At such presentations, respondents should be prepared to address any questions regarding their Expressions of Interest.
The Commonwealth requests that questions be submitted by email no later than August 23, 2019 so that staff can adequately address all questions. Questions may be sent to our Procurement Office at: email@example.com. However, it is anticipated that a limited number of additional questions will be accepted during the webinar, as well.
A listing of the written questions submitted and the Commonwealth’s responses will also be made available beginning on August 25, 2019. An addendum to the RFA will be posted at eMarketplace (http://www.emarketplace.state.pa.us) with a link.
Commonwealth Reservation of Rights: The Commonwealth reserves all rights to consider or reject any and all responses to this Request, to amend and/or reissue this Request, and to abandon and then recommence at any time, or not recommence, this process. All costs of any response to this Request and participation in any presentations to the Commonwealth prior to conclusion of any contract for services with the Commonwealth are solely the responsibility of the Respondent and the Commonwealth shall not be liable for payment of any such costs.
With Free Trial, you can:
You will have a full access to bids, website, and receive daily bid report via email and web.
Solicitation# Types Solicitation Title Description Agency County Amended Date Solicitation Start Date Solicitation
State Government of Pennsylvania
Bid Due: 12/31/2020
Jacket Number Rev. Office Name Quote Due Date (ET) Date Posted Product Type
U.S. Government Publishing Office
Bid Due: 9/18/2019
Opportunity History Original Synopsis Jun 21, 2019 3:43 pm Solicitation Number : M00264-19-Q-0130
Department of the Navy
Opportunity Description Dept/Agency Service Type Amount Opening Date Closing Date Status 21190822135847 The
City of Philadelphia
Bid Due: 9/26/2019