evaluation

=Evaluation Overview=

Introduction
//Becoming America// is required by the U.S. Department of Education to collect and analyze data about the impact of the project on teachers and their students. To meet this requirement, Dr. Diane Schilder, the independent project evaluator, will be asking //Becoming America// participants and comparison teachers to participate in the evaluation. //Note: The evaluation is not assessing individual teachers! It is only designed to assess the project itself//.

Evaluation Activities
The evaluation activities for the entire year are as follows:

- Survey 1 -- Baseline Background/Education and American History Knowledge Survey (participants completed this in late spring 2011). This survey asked questions, linked with the U.S. Department of Education requirements, to include participants' baseline American history knowledge. Survey link was provided to participants via e-mail. - Survey 2 -- Post-Summer Institute Survey of participating teachers captured teachers’ perspectives about the content and format of the Institute (September 2011) - Survey 3 -- Follow-up Teacher Surveys (spring 2012) captured changes in teachers’ education and American history content knowledge.
 * Surveys**

- Pre- and post-tests aligned with each lesson administered to students of participating teachers and //comparison students identified by participating teachers// (late winter 2011 to spring 2012, depending upon dates of implementation). Teachers are asked to contribute to the construction of these assessments by selecting assessment questions (See below) and creating any new items that relate specifically to the lesson.
 * Student assessments**

- Comparison teachers who are not participating in the project are asked to complete a web-based survey and to administer the pre- and post- student assessments. Participating comparison teachers receive a $5 Target gift card. Comparison teacher survey is here: @https://www.surveymonkey.com/s/TAHSpring2012
 * Comparison teacher data**

- During and after participation in //Becoming America// activities of participating and comparison teachers (spring 2012 through spring 2013, depending on dates of implementation).
 * Interviews, observations and informal information sharin**g

Contact Information
Dr. Schilder will be working with Teacher Learning Center Directors on these activities. If you have questions or comments, please contact Diane at 617-816-2026.

A status chart is presented below the assessment questions section.
 * Status**

**ASSESSMENT QUESTIONS**
Here is the page with the test bank of questions from which to draw: Assessment11_items

50% or more of the questions on the pre-/post-assessment should be drawn from this test bank. These questions should be used "as is." You may write your own question or modify items from the test bank for additional items for the pre-/post- assessment.

When you have selected the test bank questions you wish to use, note the numbers of those questions on your Teachers Side page under the "Assessment" section. Include any questions you have modified or written.

Diane Schilder, Project Evaluator, will prepare the pre-/post- assessment for you and email it to you. Email her at dschilder@eval-inc.com when you have your implementation dates; please include your Director on the email as well.

IMPLEMENTATION CHECKLIST (Year 3, 2012)
Year 3 - All 2010-2011 project participants must complete and submit this form.

PARTICIPATING TEACHER SURVEY
Year 3- This survey takes approximately 15 minutes.

COMPARISON TEACHER SURVEY
=== Year 3 - This survey takes approximately 15 minutes. To thank your comparison teacher for her/his help in our project, when your comparison teacher completes both this survey and administer the student assessments (pre- and post-tests) to their students (the comparison group), s/he will receive a $5.00 Target Gift Card. ===

**2011/2012 Participants Implementation Status**
Updated 1/17/2012 June 5-7 (Lesson) June 8 (Post Test) || Yes || yes ||  || Yes, Ryan Casey ||   || 6/14/2012 || May 22-May 31 (Lessons) June 1 (Post Test) || Yes || yes ||  || Yes, Sherri MacDougall ||   || 6/14/2012 || May 16-18 (Lesson) May 21 (Post Test) || Yes || Yes ||  || Yes, Comp: Traveline ||   || 6/14/2012 || May 7-9 (Lesson) May 14(Post Test) || Yes || Yes ||  || Yes ||   || 5/14/2012 || March 26, 27 28 Lesson (Cathleen and Pat Ells Observed) Week April 2 (post test) || Yes || Yes ||  || Yes :) ||   || 5/14/2012 || April 27 - 1:30-2:15 May 2 - 8:30-9:15 || BAY3 Items: 6, Modify 32 BAY2: 32 || Yes ||   || Yes - RS Comp. Teacher Marianne Dushinski ||   || 5/14/2012 || May 2 - Class 1 -10:40-11:15 May 3 - Class 2 - 10:40 - 11:15 May 4- Class 3 10:40 - 11:15 May 9 - Class 4 ? || BAY3 Items: 6,22 BAY2 Items: 3, 6 || Yes ||   || Yes - RS Comp. Teacher Margie Briatico ||   || 5/14/2012 || May 1 - class 2 1:40-2:30 May 2 - class 3 1:40-2:30 May 3 - class 4 1:40-2:30 May 4 - class 5 1:40-2:30 May 8 - class 6 1:40-2:30 final edit/podcast || NAEP 4th 32,34a.b.,37 || Yes ||   || Yes - RS Comp. Teacher Samantha Roberts ||   || 5/14/2012 || possibly Dec 2012/Jan 2013; NEXT YEAR || BAY 3 Items: 1, 2, 3, 6, 10, 11, 12, 13, 22, 25 BAY 2 Items: 1, 4, 5, 7, 9, 13 ||   ||   ||   ||   ||   || Room 8B || BAY 3 Items: 6. BAY 2 Items: 2, 3, 7,18, 32 || Yes ||   || Yes -RS Comp. Teacher Natalie Goodwin ||  || 5/2012 ||
 * ** First name ** || ** Last name ** || ** Planned implementation dates ** || **Selected assessment items** || Teacher has assessment ||  || **Lesson Fully Implemented** ||   || **Checked** ||
 * ** BRAINTREE PUBLIC SCHOOLS ** ||  ||||||||||||||   ||
 * **Luke** || **Dedeus** || May 25 || yes || yes ||  || Yes, Rebecca Drayer ||   || 6/14/2012 ||
 * **Rebecca ** || **Drayer ** || June 4 (Pre-test)
 * **Joanna ** || **Newton ** || May 21 (Pre-test)
 * **Jean** || **Witt** || NOT IMPLEMENTING ||  ||   ||   ||   ||   ||   ||
 * ** CHELSEA PUBLIC SCHOOLS ** ||  ||   ||   ||   ||   ||   ||   ||   ||
 * **Jessica** || **Travaline** || Yes || Yes || Yes ||  || Yes, Comp: Powers ||   || 6/14/2012 ||
 * **Jennifer** || **Powers** || May 14 (Pre-test)
 * **Jennifer** || **Powers** || May 14 (Pre-test)
 * **Inmaculada** || **Peters** || May 3rd (Pre-test)
 * ** EVERETT PUBLIC SCHOOLS ** ||  ||   ||   ||   ||   ||   ||   ||   ||
 * **Lisa** || **DiSalvatore** || Mar 13 (Pre-test)
 * **Lisa** || **DiSalvatore** || Mar 13 (Pre-test)
 * **Helen** || **Martin** || **June 14/June 18** || Yes || Yes ||  ||   ||   ||   ||
 * **Mary** || **Puleo** || 4/11,4/24, 5/2,5/4 || Yes || Yes ||  || Yes ||   || 5/14/2012 ||
 * ||  || 10:15-11:00 am ||   ||   ||   ||   ||   ||   ||
 * ** MALDEN PUBLIC SCHOOLS ** ||  ||   ||   ||   ||   ||   ||   ||   ||
 * **Larry** || **Higgins** || June 6, 7, 8 || BAY3 Items: 8,9,11,16,33 || Yes ||  || Yes, Leslie Tombly ||   || 6/14/2012 ||
 * **Barbara** || **Kennedy** || April 25 - 1:30-2:15
 * **Mary ** || **Liberge** || May 1 - Pretest 10:40 - 11:15
 * **Colleen** || **Synan** || April 30 - class 1 1:40-2:30
 * ** MEDFORD PUBLIC SCHOOLS ** ||  ||   ||   ||   ||   ||   ||   ||   ||
 * **Matt** || **Brophy** || June 2012 || yes || yes ||  ||   ||   ||   ||
 * **Deborah** || **Corleto** || pre test Thurs March 29;Lesson implemented Thursday April 5 @ Noon-12:40 || yes || yes ||  || Yes ||   || 5/2012 ||
 * **Kevin** || **Cullen** || Maybe June 2012 but
 * **Kevin** || **Cullen** || Maybe June 2012 but
 * **Maria** || **D'Orsi** || NEXT YEAR ||  ||   ||   ||   ||   ||   ||
 * **Ashley** || **Yoerger** || May 2012 || Yes || Yes ||  ||   ||   ||   ||
 * ** MYSTIC VALLEY REGIONAL CHARTER SCHOOL ** ||  ||   ||   ||   ||   ||   ||
 * **Matt ** || **Stone** || May 2, 3, 4 - 8:30-9:15am 10-10:45am 1-1:45pm 2:40-3:25pm
 * **Matt ** || **Stone** || May 2, 3, 4 - 8:30-9:15am 10-10:45am 1-1:45pm 2:40-3:25pm

**2010/2011 Participants Implementation Status**
Updated 3/9/2011 (Cathleen observed) || X || X ||  || X ||   || X ||
 * ** First name ** || ** Last name ** || ** Planned implementation dates ** || **Selected assessment items** || Teacher has assessment ||  || **Lesson Fully Implemented** ||   || **Checked** ||
 * ||  |||||||||||||| ** BRAINTREE PUBLIC SCHOOLS ** ||
 * Natalie || Forbes || May 16 (Meghan observed) || X || X ||  || X ||   || X ||
 * Patrick || **Condon** || May 19 (Meghan observed) || X || X ||  || X ||   || X ||
 * **Kathryn** || **Gamache** || March 28, 2011 || X || X ||  || X ||   || X ||
 * **Rachel** || **Porter** || May 5, 2011 (Meghan observed) || X || X ||  || X ||   || X ||
 * **Deborah** || **Vose** || May 5, 2011 (Meghan observed) || X || X ||  || X ||   || X ||
 * ** CHELSEA PUBLIC SCHOOLS ** ||  ||   ||   ||   ||   ||   ||   ||   ||
 * **Jessica ** || **Travaline** || May 26 (Meghan observed) || X || X ||  || X ||   || X ||
 * **Inmaculada** || **Peters** || June 2 || X || X ||  || X ||   || X ||
 * **Hope** || **Roscow** || June || X || X ||  || X ||   || ? ||
 * ** EVERETT PUBLIC SCHOOLS ** ||  ||   ||   ||   ||   ||   ||   ||   ||
 * Robert || D'Agostino || June 1, 2 (Cathleen observed) || X || X ||  || X ||   || X ||
 * **Lisa ** || **DiSalvatore ** || April 27, May 5 and May 11, 2011 (Diane observed May 5) || X || X ||  || X ||   || X ||
 * **Linda** || **Maltais** || June 2, 8, 10
 * **Lisa ** || **DiSalvatore ** || April 27, May 5 and May 11, 2011 (Diane observed May 5) || X || X ||  || X ||   || X ||
 * **Linda** || **Maltais** || June 2, 8, 10
 * **Helen ** || **Martin** || **June 14/June 18** || X || X ||  ||   ||   ||   ||
 * **Mecky** || **Napoli** || April 27 & 29, May 2 & 3, 2011 (Meghan observed) || X || X ||  || X ||   || X ||
 * **Mary** || **Puleo** || April 11, 25 2011 and May 2011 (Cathleen to observe) || X || X ||  || X ||   || X ||
 * ** MALDEN PUBLIC SCHOOLS ** ||  ||   ||   ||   ||   ||   ||   ||   ||
 * **Mike** || **Lightbody** || Fall 2011 || X || X ||  ||   ||   ||   ||
 * **Paul** || **Mallett** || 1st or 2nd week May 2011 || X || X ||  || X ||   || X ||
 * **Colleen** || **Synan** || Implemented March 2011 || X || X ||  || X ||   || X ||
 * ** MEDFORD PUBLIC SCHOOLS ** ||  ||   ||   ||   ||   ||   ||   ||   ||
 * **Deborah** || **Corleto** || After April Break || X || X ||  || X ||   || X ||
 * **Kevin** || **Cullen** || Fall 2011 || X || X ||  || X ||   ||   ||
 * **Michela** || **DeBari** || Fall 2011 || X || X ||  || X ||   || June 14, 2012 ||
 * **Maria ** || **D'Orsi** || Week of May 2 || X || X ||  || X ||   || X ||
 * **Jody** || **Liu** || Week of May 2 (Meghan observed) || X || X ||  || X ||   || X ||
 * **Kathleen** || **O'Donoghue** || Aiming for spring 2011 || X || X ||  || X ||   || X ||
 * ** MYSTIC VALLEY REGIONAL CHARTER SCHOOL ** ||  ||   ||   ||   ||   ||   ||
 * Aaron || Minton || End of April 2011 || X || X ||  || X ||   || X ||

For an Archive of previous years' Evaluation Pages, go to Evaluation_yr2 Page