Nebula Background

Evaluation Information - 2019-2020 NASA Projects

 

Welcome to the Evaluation Page. We are excited to work with you on this project. You will find information below to enable and facilitate your site’s participation in the evaluation component of this project.

Bienvenidos a la Página de Evaluación. Estamos muy contentos de trabajar con usted en este proyecto. Más abajo encontrará información que permitirá y facilitará la participación de su sitio (escuela) en el componente de evaluación de este proyecto. NOTA: Los documentos para esta evaluación se encuentran disponibles para padres/familias en español e inglés. 

 

Evaluation Frequently Asked Questions

Click to view the FAQs!

Click to view the recording of the November 21st Evaluation webinar!

 

Parental Passive Consent Notice 

Instructions: Parents will have the option to opt-out their child(ren) from participating in the STEM out-of-school/afterschool Engineering Design Challenges [EDC] data collection for the evaluation.  In order for this to occur, after being informed of the EDC program, parents should receive a copy of the consent notice describing the data collection activities that will be used in the evaluation of the program’s goals and objectives. 

Afterwards, if parent(s) do not want their child(ren) to participate in the evaluation data collection, please remind them to complete, sign, and return the handout which indicates that the parent does not want their child(ren) to participate in the data collection. Click here to download the PDF.

Nota de Consentimiento Pasivo para Familias

Si usted no desea que su estudiante participe en la recolección de información, usted debe llenar la encuesta acerca del Conocimiento de Familia en español aquí: PDF

NOTE TO FACILITATOR:  Provide each parent a passive consent notice handout. Only if the parent does not want their child(ren) to participate in the collection of evaluation data, should the parent complete the hard copy handout, sign it, and return the signed notice to the facilitator. The POC/facilitator should return the signed parental passive consent document scanned, or uploaded to the Y4Y site and returned to the evaluation team by the facilitator/point of contact (POC) via the Y4Y NASA email address .(JavaScript must be enabled to view this email address) as soon as possible.  Please include the following contact information on your site’s cover sheet to identify your site: Name, State, Site Name, EDC Start Date, Name of Facilitator (teacher, instructor), Email Address, and Telephone Number. The student survey should not be administered to students whose parents do not want them to participate in data collection activities.

 

Parental Active Consent Notice

Instructions:  The parental active consent notice is only for sites required by their state to provide written consent for their child’s participation in research/evaluation activities.  If you are unsure of which consent form is required from your state, contact your state education agency (SEA).  This consent notice is to be provided directly to parents as a handout.  Parents must complete this parental active consent by signing and approving their child(ren) to participate in the evaluation data collection activities.  If your state requires active consent for parents, please have parent(s) complete, sign, and return the handout which indicates that the parent approves their child(ren) to participate in the data collection.  Click here to download the PDF.

Nota de Consentimiento de las Familias

Consentimiento de los padres para la evaluación de datos del estudiante: PDF

NOTE TO FACILITATOR:  If required the by State/District, provide each parent an active consent notice handout.  It is mandatory that each parent who wants their child(ren) to participate in the collection of evaluation data, complete the hard copy form, sign it, return the signed notice to the facilitator.  The POC/facilitator should return the signed parental active consent document scanned, or uploaded to the Y4Y site and returned to the evaluation team by the facilitator/point of contact (POC) via the Y4Y NASA email address .(JavaScript must be enabled to view this email address) as soon as possible.  Please include the following contact information on your site’s cover sheet to identify your site:  Name, State, Site Name, EDC Start Date, Name of Facilitator (teacher, instructor), Email Address, and Telephone Number.  The student survey should not be administered to students whose parents have not completed, signed and returned the active consent in order for them to participate in data collection activities.

 

Parent Survey

Instructions:  All parents with children in the EDC program will be asked to complete a parent survey.  This parent survey questionnaire is part of the data collection needed for the evaluation of the NASA and US Department of Education 21st Century Community Learning Collaboration Phase 4.  The goal of the survey is to obtain parental views on parental engagement and interest in Science, Technology, Engineering, and Mathematics (STEM) activities and education for their child(ren).  All information obtained is confidential. The information provided will be used to enhance parental engagement in the delivery of children’s mathematics and science education to support students.  We ask that the facilitator working with the students provide this SurveyMonkey link below.

NOTE TO FACILITATOR:  For those parent(s) requiring a handout, please collect completed surveys, scan, and return to the evaluation team via the Y4Y NASA email address Y4YNASA@seiservices.com as soon as possible. Please include the following contact information on your site’s cover sheet to identify your site:  Name, State, Site Name, EDC Start Date, Name of Facilitator (teacher, instructor), Email Address, and Telephone Number.

 

Student Surveys

Instructions: All sites with children in the EDC program will complete a student survey. The student survey questionnaire is part of the data collection needed for the evaluation of the NASA and US Department of Education 21st Century Community Learning Collaboration Phase 4. The goal of the survey is to obtain student views on STEM engagement, interest, and motivation, activities and knowledge of STEM career options and pathways. All information obtained is confidential. The information provided will be used to enhance student STEM engagement in the delivery of children’s mathematics and science education to support students. We ask that the facilitator working with the students provide the SurveyMonkey link for grades 3 to 8.

 

Retrospective Survey (Students’ Attitudes toward STEM (S-STEM) Survey)

For Students in Grades 3 to 8

Facilitator Instructions:  

We are excited to learn from your students about their perceptions and attitudes regarding topics related to STEM learning and their opinions about the impact of the Engineering Design Challenge (EDC) regarding their STEM confidence.

Facilitators should provide each student in grades 3 to 8 a retrospective survey after implementation of the EDC challenges.  Prior to starting the retrospective survey, each student must be provided a unique student number from the program evaluators.  Contact the Y4Y NASA Help Desk, if you require assistance.

For the purpose of ease, the retrospective survey should be administered to students in an instructor led approach at the completion of the class/course.  This will help to ensure that students spend ample time on each question and understand what is being asked of them as a part of their response.  Depending on your perception of the reading level of the students, you may have to read the survey to the students. You may have to explain some of the terms to the students (e.g., technology) to the students.

Since we want to track how long it takes to complete the survey, please indicate to the students the time that the survey is started.  Once a student completes a survey, we ask that you provide them with the time so that it can be recorded on the survey by the students.

Please be sure to remind the students to use their unique identifying number. The surveys can be completed on a computer, handheld devices, or with paper and pencil.  When students use paper copies to complete the survey, please scan them and forward them to the evaluation team via the Y4Y NASA email address Y4YNASA@seiservices.com.   

Retrospective Survey:  https://www.surveymonkey.com/r/Retrospectivesurvey3-8grades  OR  Click here to download the PDF.

Encuesta Retrospectiva:  https://www.surveymonkey.com/r/Spanishretrospectivesurvey3to8g  |  PDF

NOTE TO FACILITATOR:  For those student(s) completing the survey using a hard copy form, that document should be scanned and returned to the evaluation team by the facilitator/point of contact to via the Y4Y NASA email address Y4YNASA@seiservices.com as soon as possible.  Please include the following contact information on your site’s cover sheet to identify your site:  Name, State, Site Name, EDC Start Date, Name of Facilitator (teacher, instructor), Email Address, and Telephone Number.

 

Evaluation Administrative Check-in Instructions to Facilitators/Points of Contacts

Instructions:  We are requesting facilitators/POCs to complete bi-monthly administrative check-ins on SurveyMonkey during the Engineering Design Challenges (EDC) implementation. 

Thank you for participating in the NASA and Department of Education 21st Century Community Learning Collaboration Phase 4 program.  We are so excited that you joined us for this journey.  In an effort to learn more about each site and support everyone more effectively, we are looking to gather information through administrative check-ins.  Administrative check-ins are opportunities for sites to share their experiences and provide important information relating to their sites. 

Administrative Check-ins: https://www.surveymonkey.com/r/nasaadmincheckin

NOTE TO FACILITATOR:  As a part of the evaluation process, each site should keep attendance logs.  We will contact each site facilitator/POC to gather participant attendance. 

How will we collect data? In an effort to gather participant (student) attendance from each site, we are asking that the POC/facilitator provide their students’ attendance on the Excel participant (student) attendance spreadsheet provided to your POC/facilitator email address (starting this week) from the evaluation team via emailThis participant attendance Excel spreadsheet should be done for each class/session held according to your site’s attendance logs. This spreadsheet should be returned at the end of EDC implementation via the Help Desk at the Y4Y webpage: .(JavaScript must be enabled to view this email address).

Also, please note that the assigned student unique identifier number should be provided to the same student for the retrospective survey prior to the administration of the retrospective survey at the end of the EDC implementation.

Please include in the subject line the following phrase: “Attention Evaluation Participant Attendance”.  As a continual part of the evaluation process, each site should be keeping attendance logs at their site.

 

Next Steps After Receiving your Participant (Student) Attendance from the Evaluation Team

Please note that the student unique identifier number (UIN) is pre-populated on the participant (student) attendance form.  Mark (P) for present and (A) for absent.  Each site should match each student name with a student unique identifier number (UIN) and provide the evaluation team only the marked attendance of presence or absence on the participant (student) attendance form.  Student names should NOT be included on the spreadsheet. Once completed, send the .(JavaScript must be enabled to view this email address) Help Desk – Attention Evaluation Participant Attendance. 

Please see Participant (Student) Attendance Example below.

 

Retrospective Student Unique Identifier Number (UIN)

Prior to starting the retrospective survey, each student must be provided a unique student number UIN from the program evaluators.  Each site should match the students’ name with a student UIN on-site and provide the evaluation team only the marked attendance of presence or absent on the participant (student) attendance form.  Each student will use the student UIN on the retrospective survey that will be administered one-time at the end of the EDC implementation.

 

Participant Attendance Excel Form Instructions

Each site will be given a Participant (Student) Attendance form from the evaluation team via email directly to the POC/facilitator that includes the following information:

Name of Site – For example, ABC Boys and Girls Club in Florida.

Site and State Number- Site Number and State Initial, i.e. 01FL - Florida Site 1; 02FL - Florida Site 2, and so on.

Student Unique Identifier Numbers (UIN) - Student participant (i.e. FL001, FL002, FL003, and so on).

EDC Dates of Implementation by Session/Class:  Enter the EDC implementation date(s) under each session/class column and for each row (site number, state, student) select in the drop down box (P) for present or (A) for absent.  If you have additional session/classes beyond 10, let us know and we will give you custom or a unique spreadsheet.  See example below.

 

At the end of the EDC implementation, send the completed participant (student) attendance to the Y4YNASA@seiservices.com Help Desk – Attention Evaluation Participant Attendance. 

 

Facilitator/Instructor Surveys

Each facilitator will complete the following two facilitator surveys at the end of the EDC implementation; 1) Post-Facilitator Confidence Survey and the Post-EDC Facilitator/Instructor Activity Educator Feedback Form and the EDC implementation. Prior to starting the two surveys, each facilitator must be provided a unique 4-digit number from the program evaluators.  Contact the Y4Y NASA email address Y4YNASA@seiservices.com, if you require assistance.

 

1. Post-Facilitator Confidence Survey

Instructions for the Post-Facilitator Confidence SurveyAt the end of implementation, we are asking that each facilitator complete the post-facilitator confidence survey and share with us how confident you are instructing and motivating students NOW following the Professional Development (PD)Training in the NASA and U.S. Department of Education 21st CCLC. 

Facilitator/Instructor Confidence Survey:  https://www.surveymonkey.com/r/postfacilitatorconfidencesurvey OR Click here to download the PDF.

NOTE TO FACILITATOR:  For those facilitators (s) completing the survey using a hard copy, that document should be scanned and returned to the evaluation team by the facilitator/point of contact to via the Y4Y NASA email address Y4YNASA@seiservices.com as soon as possible.  Please include the following contact information on your site’s cover sheet to identify your site:  Name, State, Site Name, EDC Start Date, Name of Facilitator (teacher, instructor), Email Address, and Telephone Number.

 

2. Facilitator/Instructor Survey (Post-EDC Facilitator/Instructor Activity Educator Feedback Form)

Instructions for the Facilitator/Instructor Post-EDC Facilitator/Instructor Activity Educator Feedback Form:  At the end of the culminating event, we are asking that each facilitator complete the activity educator feedback form.  As an instructor of a recently completed NASA & US Department of Education 21st Century Community Learning Collaboration Phase 4 Engineering Design Challenge (EDC) Activity, we are asking that you take part in this questionnaire to evaluate your experiences with the NASA EDC you instructed.  Your answers to these questions will help us learn more about the activity, including what worked well and what we may improve on. 

Facilitator/Instructor Post-PD Culminating Event Survey:  https://www.surveymonkey.com/r/facilitatorinstructorpostpdculminatingevent OR Click here to download the PDF.

NOTE TO FACILITATOR:  For those facilitators (s) completing the survey using a hard copy, that document should be scanned and returned to the evaluation team by the facilitator/point of contact to via the Y4Y NASA email address Y4YNASA@seiservices.com as soon as possible.  Please include the following contact information on your site’s cover sheet to identify your site:  Name, State, Site Name, EDC Start Date, Name of Facilitator (teacher, instructor), Email Address, and Telephone Number.

 

Evaluation Overview Presentation

Evaluation Webinar

Information:  This mandatory evaluation webinar training provided the sites/facilitators the critical information to complete participation in evaluation activities, usage of the unique identifier numbers for facilitators and students, and timely submission of data collection activities.  Once sites were registered with Y4Y, the evaluation webinar training dates were available on August 13, September 18, and October 9 (10AM or 3PM EST).   It was important for sites to participate in this evaluation webinar training to understand the role of facilitating evaluation activities at the site to support the documentation and assessment of the program. 

We understand that due to scheduling conflicts, participation by some sites in the webinars may be difficult.  Click here to view a recording of the Evaluation Webinar

 

Evaluation Questions

All communications regarding evaluation should start with an email to: .(JavaScript must be enabled to view this email address)

NOTE: Frequently Asked Questions (FAQs) are available on the Y4Y NASA webpage to support to any common evaluation questions.  If you have a specific question, comment, or concern, contact the Y4Y NASA Help Desk.

Email Instructions:  In the subject line of the email please enter Evaluation followed by the topic (e.g., Subject: Evaluation General Question; Evaluation Parent Surveys; Evaluation New Facilitator ID; etc.)

 

Frequently Asked Questions (FAQs)

How does a facilitator receive a new unique identifier number (UIN), a four-digit number? 

Please email Y4YNASA@seiservices.com Help Desk for a new UID so the facilitator can complete the appropriate surveys.

When you contact the Help Desk, provide the following information in your email message:

  • Subject: Evaluation Question UIN
  • Name
  • State
  • Site Name
  • Name of Facilitator (teacher, instructor)
  • Email Address
  • Telephone Number

Once the evaluation team receives the detailed information, we will respond to your question and/or provide you with the requested information within normally 3 business days.

 

How does a facilitator obtain a forgotten unique identifier number (UIN), a four-digit number, from the professional development trainings?

If you are unable to locate your UIN given at the professional development face-to-face trainings, please email Y4YNASA@seiservices.com Help Desk for a UID so the facilitator can complete the appropriate surveys. 

When you contact the Help Desk, provide the following information in your email message:

Subject: Evaluation Question UIN

  • Name
  • State
  • Site Name
  • Name of Facilitator (teacher, instructor)
  • Email Address
  • Telephone Number

Once the evaluation team receives the detailed information, we will respond to your question and/or provide you with the requested information within normally 3 business days.

 

Evaluation Keys to Success

  • Review the mandatory evaluation webinar training recording
    • See the evaluation webinar section for archived recording
  • Participate in evaluation activities
    • Instruments/surveys, administrative check-ins, and EDC culminating project videos or slide presentations
    • As a part of the evaluation process, each site should keep attendance logs.  We will contact each site facilitator/POC to gather participant attendance. 
  • Usage of unique identifier numbers for facilitators and students
    • Important to track participation in surveys for program feedback
  • Timely submission of data collection activities
    • Ensure site profile information (implementation start and end date) are accurate and up-to-date
    • Instruments/surveys, administrative check-ins, and EDC culminating project videos or slide presentations