Evaluation Overview

Introduction

Becoming America is required by the U.S. Department of Education to collect and analyze data about the impact of the project on teachers and their students. To meet this requirement, Dr. Diane Schilder, the independent project evaluator, will be asking Becoming America participants and comparison teachers to participate in the evaluation. Note: The evaluation is not assessing individual teachers! It is only designed to assess the project itself.

Evaluation Activities

The evaluation activities for the entire year are as follows:

Surveys
- Survey 1 -- Baseline Background/Education and American History Knowledge Survey (participants completed this in late spring 2011). This survey asked questions, linked with the U.S. Department of Education requirements, to include participants' baseline American history knowledge. Survey link was provided to participants via e-mail.
- Survey 2 -- Post-Summer Institute Survey of participating teachers captured teachers’ perspectives about the content and format of the Institute (September 2011)
- Survey 3 -- Follow-up Teacher Surveys (spring 2012) captured changes in teachers’ education and American history content knowledge.


Student assessments
- Pre- and post-tests aligned with each lesson administered to students of participating teachers and comparison students identified by participating teachers (late winter 2011 to spring 2012, depending upon dates of implementation). Teachers are asked to contribute to the construction of these assessments by selecting assessment questions (See below) and creating any new items that relate specifically to the lesson.

Comparison teacher data
- Comparison teachers who are not participating in the project are asked to complete a web-based survey and to administer the pre- and post- student assessments. Participating comparison teachers receive a $5 Target gift card.
Comparison teacher survey is here:
https://www.surveymonkey.com/s/TAHSpring2012

Interviews, observations and informal information sharing
- During and after participation in Becoming America activities of participating and comparison teachers (spring 2012 through spring 2013, depending on dates of implementation).

Contact Information

Dr. Schilder will be working with Teacher Learning Center Directors on these activities. If you have questions or comments, please contact Diane at 617-816-2026.

Status
A status chart is presented below the assessment questions section.


ASSESSMENT QUESTIONS

Here is the page with the test bank of questions from which to draw:
Assessment11_items

50% or more of the questions on the pre-/post-assessment should be drawn from this test bank. These questions should be used "as is."
You may write your own question or modify items from the test bank for additional items for the pre-/post- assessment.

When you have selected the test bank questions you wish to use, note the numbers of those questions on your Teachers Side page under the "Assessment" section. Include any questions you have modified or written.

Diane Schilder, Project Evaluator, will prepare the pre-/post- assessment for you and email it to you. Email her at dschilder@eval-inc.com when you have your implementation dates; please include your Director on the email as well.




IMPLEMENTATION CHECKLIST (Year 3, 2012)

Year 3 - All 2010-2011 project participants must complete and submit this form.

PARTICIPATING TEACHER SURVEY


Year 3- This survey takes approximately 15 minutes.


COMPARISON TEACHER SURVEY

Year 3 - This survey takes approximately 15 minutes. To thank your comparison teacher for her/his help in our project, when your comparison teacher completes both this survey and administer the student assessments (pre- and post-tests) to their students (the comparison group), s/he will receive a $5.00 Target Gift Card.




2011/2012 Participants Implementation Status

Updated 1/17/2012
First name
Last name
Planned implementation dates
Selected assessment items
Teacher has assessment

Lesson Fully Implemented

Checked
BRAINTREE PUBLIC SCHOOLS


Luke
Dedeus
May 25
yes
yes

Yes, Rebecca Drayer

6/14/2012
Rebecca
Drayer
June 4 (Pre-test)
June 5-7 (Lesson)
June 8 (Post Test)
Yes
yes

Yes, Ryan Casey

6/14/2012
Joanna
Newton
May 21 (Pre-test)
May 22-May 31 (Lessons)
June 1 (Post Test)
Yes
yes

Yes, Sherri MacDougall

6/14/2012
Jean
Witt
NOT IMPLEMENTING















CHELSEA PUBLIC SCHOOLS








Jessica
Travaline
Yes
Yes
Yes

Yes, Comp: Powers

6/14/2012
Jennifer
Powers
May 14 (Pre-test)
May 16-18 (Lesson)
May 21 (Post Test)
Yes
Yes

Yes, Comp: Traveline

6/14/2012
Inmaculada
Peters
May 3rd (Pre-test)
May 7-9 (Lesson)
May 14(Post Test)
Yes
Yes

Yes

5/14/2012









EVERETT PUBLIC SCHOOLS








Lisa
DiSalvatore
Mar 13 (Pre-test)
March 26, 27 28 Lesson (Cathleen and Pat Ells Observed)
Week April 2 (post test)
Yes
Yes

Yes :)

5/14/2012
Helen
Martin
June 14/June 18
Yes
Yes




Mary
Puleo
4/11,4/24, 5/2,5/4
Yes
Yes

Yes

5/14/2012


10:15-11:00 am






MALDEN PUBLIC SCHOOLS








Larry
Higgins
June 6, 7, 8
BAY3 Items: 8,9,11,16,33
Yes

Yes, Leslie Tombly

6/14/2012
Barbara
Kennedy
April 25 - 1:30-2:15
April 27 - 1:30-2:15
May 2 - 8:30-9:15
BAY3 Items: 6, Modify 32 BAY2: 32
Yes

Yes - RS
Comp. Teacher
Marianne Dushinski

5/14/2012
Mary
Liberge
May 1 - Pretest 10:40 - 11:15
May 2 - Class 1 -10:40-11:15
May 3 - Class 2 - 10:40 - 11:15
May 4- Class 3 10:40 - 11:15
May 9 - Class 4 ?
BAY3 Items: 6,22 BAY2 Items: 3, 6
Yes

Yes - RS
Comp. Teacher
Margie Briatico

5/14/2012
Colleen
Synan
April 30 - class 1 1:40-2:30
May 1 - class 2 1:40-2:30
May 2 - class 3 1:40-2:30
May 3 - class 4 1:40-2:30
May 4 - class 5 1:40-2:30
May 8 - class 6 1:40-2:30 final edit/podcast
NAEP 4th 32,34a.b.,37
Yes

Yes - RS
Comp. Teacher
Samantha Roberts

5/14/2012









MEDFORD PUBLIC SCHOOLS








Matt
Brophy
June 2012
yes
yes




Deborah
Corleto
pre test Thurs March 29;Lesson implemented Thursday April 5 @ Noon-12:40
yes
yes

Yes

5/2012
Kevin
Cullen
Maybe June 2012 but
possibly Dec 2012/Jan 2013; NEXT YEAR
BAY 3 Items: 1, 2, 3, 6, 10, 11, 12, 13, 22, 25
BAY 2 Items: 1, 4, 5, 7, 9, 13





Maria
D'Orsi
NEXT YEAR






Ashley
Yoerger
May 2012
Yes
Yes













MYSTIC VALLEY REGIONAL CHARTER SCHOOL






Matt
Stone
May 2, 3, 4 - 8:30-9:15am 10-10:45am 1-1:45pm 2:40-3:25pm
Room 8B
BAY 3 Items: 6. BAY 2 Items: 2, 3, 7,18, 32
Yes

Yes -RS
Comp. Teacher
Natalie Goodwin

5/2012




2010/2011 Participants Implementation Status

Updated 3/9/2011
First name
Last name
Planned implementation dates
Selected assessment items
Teacher has assessment

Lesson Fully Implemented

Checked


BRAINTREE PUBLIC SCHOOLS
Natalie
Forbes
May 16 (Meghan observed)
X
X

X

X
Patrick
Condon
May 19 (Meghan observed)
X
X

X

X
Kathryn
Gamache
March 28, 2011
X
X

X

X
Rachel
Porter
May 5, 2011 (Meghan observed)
X
X

X

X
Deborah
Vose
May 5, 2011 (Meghan observed)
X
X

X

X









CHELSEA PUBLIC SCHOOLS








Jessica
Travaline
May 26 (Meghan observed)
X
X

X

X
Inmaculada
Peters
June 2
X
X

X

X
Hope
Roscow
June
X
X

X

?









EVERETT PUBLIC SCHOOLS








Robert
D'Agostino
June 1, 2 (Cathleen observed)
X
X

X

X
Lisa
DiSalvatore
April 27, May 5 and May 11, 2011 (Diane observed May 5)
X
X

X

X
Linda
Maltais
June 2, 8, 10
(Cathleen observed)
X
X

X

X
Helen
Martin
June 14/June 18
X
X




Mecky
Napoli
April 27 & 29, May 2 & 3, 2011 (Meghan observed)
X
X

X

X
Mary
Puleo
April 11, 25 2011 and May 2011 (Cathleen to observe)
X
X

X

X
MALDEN PUBLIC SCHOOLS








Mike
Lightbody
Fall 2011
X
X




Paul
Mallett
1st or 2nd week May 2011
X
X

X

X
Colleen
Synan
Implemented March 2011
X
X

X

X
MEDFORD PUBLIC SCHOOLS








Deborah
Corleto
After April Break
X
X

X

X
Kevin
Cullen
Fall 2011
X
X

X


Michela
DeBari
Fall 2011
X
X

X

June 14, 2012
Maria
D'Orsi
Week of May 2
X
X

X

X
Jody
Liu
Week of May 2 (Meghan observed)
X
X

X

X
Kathleen
O'Donoghue
Aiming for spring 2011
X
X

X

X
MYSTIC VALLEY REGIONAL CHARTER SCHOOL






Aaron
Minton
End of April 2011
X
X

X

X



For an Archive of previous years' Evaluation Pages, go to Evaluation_yr2 Page