Florida Regulations 6A-5.0411: Calculations of Student Learning Growth for Use in School Personnel Evaluations
Current as of: 2024 | Check for updates
|
Other versions
(1) Purpose. The purpose of this rule is to provide districts choosing to use the formulas for measuring student learning growth approved by the commissioner with a framework for using and interpreting scores in school district instructional personnel evaluation systems developed under Florida Statutes § 1012.34, and to provide information for use in the approval of school leader preparation programs under Florida Statutes § 1012.562
(2) Definitions. For the purposes of this rule, the following definitions apply.
(a) “”Confidence interval.”” A confidence interval is derived from the standard error. It expresses the precision of a statistic as a range of values. An individual teacher’s VAM score is an estimate of that teacher’s contributions to student learning growth. The 95% confidence interval used in classification represents a range of possible values that would include the teacher’s VAM score 95% of the time if VAM scores were repeatedly re-estimated with different students for each teacher.
(b) “”Courses associated with statewide, standardized assessments”” or “”courses associated with statewide, standardized assessments under Florida Statutes § 1008.22,”” are those courses which are assessed by statewide, standardized assessments and are listed in the publication, “”Florida VAM Course List,”” (effective March 2022), which is incorporated herein by reference (http://www.flrules.org/Gateway/reference.asp?No=Ref-14061). A copy of the Florida VAM Course List may be obtained from the Florida Department of Education, 325 West Gaines Street, Room 544, Tallahassee, FL 32399-0400.
(c) “”Covariate.”” A covariate is a variable or set of variables reflecting measured characteristics used in computing a statistical model that controls for specific influences on the outcome being modeled.
(d) “”Covariate adjustment model.”” A covariate adjustment model is a statistical model that controls for the influence of one or more of the covariates.
(e) “”Expected score.”” An expected score generated by a value-added model for a statewide, standardized assessment is based on the student’s prior statewide, standardized assessment score history and covariates, as well as how other students in the state actually performed on the assessment. For each individual student, the expected score is the sum across all covariates of the value of the covariate multiplied by that covariate’s contribution to student learning as estimated by the covariate adjustment model.
(f) “”Observed score.”” An observed score is the actual score a student received on an assessment.
(g) “”Staff information system.”” The staff information system is the comprehensive management information system maintained by the Department containing staff data reported by school districts in accordance with Fl. Admin. Code R. 6A-1.0014, (http://www.flrules.org/Gateway/reference.asp?No=Ref-05723) which is incorporated by reference herein. A copy of Fl. Admin. Code R. 6A-1.0014, may be obtained from the Florida Department of Education, 325 West Gaines Street, Room 544, Tallahassee, FL 32399-0400.
(h) “”Standard error.”” A standard error is a measure of the precision of a statistic. It is determined by both sample size and sample variability.
(i) “”Student information system.”” The student information system is the comprehensive management information system maintained by the Department containing student data reported by school districts in accordance with Fl. Admin. Code R. 6A-1.0014, as incorporated in paragraph (2)(g) of this rule.
(j) “”Value-added model”” or “”VAM.”” A value-added model is a statistical model used for the purpose of determining an individual teacher’s contribution to student learning growth.
(3) Formulas for measuring student learning growth.
(a) The English Language Arts and Mathematics value-added models.
1. The formula for measuring student learning growth using student English Language Arts and Mathematics results approved by the commissioner is a covariate adjustment value-added model.
The value-added model statistically establishes the expected learning growth for each student, called an expected score. When a student’s actual performance differs from these expectations, a portion of that difference is attributed to the teacher’s and a portion is attributed to the school’s influence. Together, this information is used to compute a teacher’s value-added score. School value-added scores are the average of the teacher value-added scores within the school, and are provided to districts that choose to use them for the performance of students component of administrator evaluations required under Section 1012.34(3)(a)1., F.S.
2. The formula for the model. A full technical description of the data sources, formula, covariates, and methodology for calculating VAM scores is provided in the publication, “”Florida VAM Methodology”” (Effective August 2015) (http://www.flrules.org/Gateway/reference.asp?No=Ref-05724), which is incorporated herein by reference. A copy of “”Florida VAM Methodology”” may be obtained from the Florida Department of Education, 325 West Gaines Street, Room 544, Tallahassee, FL 32399-0400.
3. The covariates included in the value-added model approved by the commissioner are:
a. The number of subject-relevant courses in which a student is enrolled.
b. At least one (1) and up to two (2) prior years of achievement scores on the applicable statewide, standardized assessment for each student.
c. A student’s disabilities. The disabilities used within the model are limited to language impaired; deaf or hard of hearing; visually impaired; emotional/behavioral disabilities; specific learning disability; dual sensory impaired; autism spectrum disorder; traumatic brain injured; other health impaired; and other intellectual disability.
d. A student’s English Language Learner (ELL) status. This covariate is used to control for effects related to whether a student is an English language learner and has been receiving English for Speakers of Other Languages (ESOL) services for less than two (2) years; at least two (2) years but less than four (4) years; at least four (4) years but less than six (6) years; or six (6) years or longer.
e. A student’s gifted status.
f. Student attendance.
g. Student mobility. This covariate is used to control for effects related to changing schools during the school year.
h. Difference from modal age in grade. This covariate is used to control for effects related to differences in a student’s age from the most common age for students enrolled in the same grade across the state.
i. Class size. This covariate is used to control for effects related to the number of students in a class.
j. Homogeneity of students’ entering test scores in the class. This covariate is used to control for the variation in student proficiency within a classroom at the beginning of the year.
4. The formula produces a value-added score for a teacher. For English Language Arts and Mathematics, this value-added score consists of two (2) parts:
a. The teacher effect. The teacher effect is an estimate of a teacher’s contributions to student achievement as measured by scores on statewide, standardized assessments. It is based on the difference between expected scores and actual scores for a teacher’s students relative to other teachers in the school, among students assessed in the same subject at the same grade level during the same year.
b. The school component. The school component is an estimate of the part of a student’s performance that is common to students within a school. It is based on the difference between expected scores and actual scores for the school’s students relative to other schools in the state, among students assessed in the same subject at the same grade level during the same year. It represents school-level factors influencing performance of all students in a school among students assessed in the same subject at the same grade level during the same year. Fifty (50) percent of the school component shall be added to the teacher effect to create the teacher’s value-added score.
(b) The Algebra I value-added models.
1. The formula for measuring student learning growth using student results from the statewide, standardized end-of-course assessment in Algebra I pursuant to Florida Statutes § 1008.22, approved by the commissioner is a covariate adjustment value-added model.
The value-added model statistically establishes the expected learning growth for each student, called an expected score. When a student’s actual performance differs from these expectations, a portion of that difference is attributed to the teacher’s and a portion is attributed to the school’s influence. Together, this information is used to compute a teacher’s value-added score. School value-added scores are the average of the teacher value-added scores within the school, and are provided for districts that choose to use them for the performance of students component of administrator evaluations required under Section 1012.34(3)(a)1., F.S.
2. The formula for the model. A full technical description of the data sources, formula, covariates, and methodology for calculating VAM scores is provided in the publication, Florida VAM Methodology.
3. The covariates included in the Algebra I value-added model approved by the commissioner include those listed under sub-subparagraphs (3)(a)3.a.-j., as well as the following:
a. Mean prior test score. Mean prior test score is the average of the most recent prior score on the statewide, standardized assessment in Mathematics for all students within the class.
b. Percent of students in the class who are reported in the student information system as Gifted.
c. Percent at modal age in grade. Percent at modal age in grade is the percentage of students in the class whose age on September 1 of the school year is the same as the modal age of all students in the same grade.
4. The formula produces a value-added score for a teacher. For Algebra I, the score is the teacher effect. The teacher effect is an estimate of a teacher’s contributions to student achievement as measured by scores on statewide, standardized assessments. It is based on the difference between expected scores and actual scores for a teacher’s students relative to other teachers in the school, among students assessed in the same subject at the same grade level during the same year.
(4) Data Collected and Reported for VAM.
(a) Data collection and reporting procedures for purposes of VAM calculations shall be as provided in the publication, Florida VAM Methodology. As set forth in this publication in more detail, data from the Student Information System and Staff Information System obtained from Surveys 2 and 3 are used in VAM calculations. School districts submit Survey 2 and 3 data to the Department’s Student Information System and Staff Information System pursuant to Fl. Admin. Code R. 6A-1.0014 (Comprehensive Management Information System) and Fl. Admin. Code R. 6A-1.0451 (Florida Education Finance Program Student Membership Surveys) (http://www.flrules.org/Gateway/reference.asp?No=Ref-05725). These rules are incorporated by reference herein, and a copy of the rules may be obtained from the Florida Department of Education, 325 West Gaines Street, Room 544, Tallahassee, FL 32399-0400.
(b) Results provided to districts shall include the following information for each statewide, standardized assessment for which a formula has been adopted:
1. A value-added score for each teacher and administrator based on the statewide, standardized assessment associated with the course(s) that the teacher taught during the current year or the school the administrator was assigned to during the current year. This score shall be reported for each grade level and subject area covered by the statewide assessment.
2. Three-year aggregate value-added scores for each teacher, which includes data for the teacher from the current school year and each of the two (2) prior years for which data are available, for a total of at least one (1) and up to three (3) years of data for the teacher, as follows:
a. Three-year aggregate English Language Arts score. A combination of all value-added results for the teacher from all grades and courses associated with the statewide, standardized assessments in English Language Arts.
b. Three-year aggregate Mathematics score. A combination of all value-added results for the teacher from all grades and courses associated with the statewide, standardized assessments in Mathematics.
c. Three-year aggregate combined score. A combination of all value-added results associated with the statewide, standardized assessments in English Language Arts and Mathematics.
d. The aggregate score shall be calculated by standardizing the value-added scores by converting them to a proportion of a year’s average growth within the grade and subject for the year, and combining them across all grades and subjects for as many of the last three (3) years as data are available. For districts choosing to use school score(s) for administrator evaluations, districts may elect to combine multiple one (1) year, two (2) year, and three (3) year aggregate scores that reflect the school(s) to which the administrator was assigned and the years to which they were assigned to those school(s) during the period.
3. The standard error for each value-added score.
4. For each value-added score that contains a school component, the school component and teacher component reported separately.
(5) Classifying and Interpreting Scores.
(a) Data elements used by the department to classify and interpret scores are as follows:
1. The statewide average year’s growth for students in each grade and subject. For each student learning growth formula, an average year’s growth for students across the state on the statewide assessment is calculated, and once standardized, uses a threshold of zero (0) to establish performance expectations. A score of zero (0) indicates that a teacher’s students scored no higher or lower, on average, than expected.
2. The educator’s value-added score.
3. The confidence interval. A confidence interval is computed using the standard error associated with the educator’s value-added score.
(b) Ratings. Districts may use the rating provided by the department for the performance of students criterion in performance evaluations under Florida Statutes § 1012.34, for classroom teachers of courses associated with statewide, standardized assessments and administrators.
1. Ratings for Florida’s value-added models. The Department provides ratings for grade-specific and aggregate scores from the English Language Arts, Mathematics, and Algebra I value-added models.
2. The ratings for the English Language Arts, Mathematics, and Algebra I value-added models are as follows:
a. Highly Effective. A highly effective rating is demonstrated by a value-added score of greater than zero (0), where all of the scores contained within the associated 95-percent confidence interval also lie above zero (0).
b. Effective. An effective rating is demonstrated by the following:
(I) A value-added score of zero (0);
(II) A value-added score of greater than zero (0), where some portion of the range of scores associated with a 95-percent confidence interval lies at or below zero (0); or
(III) A value-added score of less than zero (0), where some portion of the range of scores associated with both the 68-percent and the 95-percent confidence interval lies at or above zero (0).
c. Needs Improvement, or Developing if the teacher has been teaching for fewer than three (3) years. A needs improvement or developing rating is demonstrated by a value-added score that is less than zero (0), where the entire 68-percent confidence interval falls below zero (0), but where a portion of the 95-percent confidence interval lies above zero (0).
d. Unsatisfactory. An unsatisfactory rating is demonstrated by a value-added score of less than zero (0), where all of the scores contained within the 95-percent confidence interval also lie below zero (0).
Rulemaking Authority 1012.34 FS. Law Implemented Florida Statutes § 1012.34. History-New 9-9-15, Amended 4-30-18, 3-15-22.
(a) “”Confidence interval.”” A confidence interval is derived from the standard error. It expresses the precision of a statistic as a range of values. An individual teacher’s VAM score is an estimate of that teacher’s contributions to student learning growth. The 95% confidence interval used in classification represents a range of possible values that would include the teacher’s VAM score 95% of the time if VAM scores were repeatedly re-estimated with different students for each teacher.
(b) “”Courses associated with statewide, standardized assessments”” or “”courses associated with statewide, standardized assessments under Florida Statutes § 1008.22,”” are those courses which are assessed by statewide, standardized assessments and are listed in the publication, “”Florida VAM Course List,”” (effective March 2022), which is incorporated herein by reference (http://www.flrules.org/Gateway/reference.asp?No=Ref-14061). A copy of the Florida VAM Course List may be obtained from the Florida Department of Education, 325 West Gaines Street, Room 544, Tallahassee, FL 32399-0400.
(c) “”Covariate.”” A covariate is a variable or set of variables reflecting measured characteristics used in computing a statistical model that controls for specific influences on the outcome being modeled.
(d) “”Covariate adjustment model.”” A covariate adjustment model is a statistical model that controls for the influence of one or more of the covariates.
(e) “”Expected score.”” An expected score generated by a value-added model for a statewide, standardized assessment is based on the student’s prior statewide, standardized assessment score history and covariates, as well as how other students in the state actually performed on the assessment. For each individual student, the expected score is the sum across all covariates of the value of the covariate multiplied by that covariate’s contribution to student learning as estimated by the covariate adjustment model.
(f) “”Observed score.”” An observed score is the actual score a student received on an assessment.
(g) “”Staff information system.”” The staff information system is the comprehensive management information system maintained by the Department containing staff data reported by school districts in accordance with Fl. Admin. Code R. 6A-1.0014, (http://www.flrules.org/Gateway/reference.asp?No=Ref-05723) which is incorporated by reference herein. A copy of Fl. Admin. Code R. 6A-1.0014, may be obtained from the Florida Department of Education, 325 West Gaines Street, Room 544, Tallahassee, FL 32399-0400.
(h) “”Standard error.”” A standard error is a measure of the precision of a statistic. It is determined by both sample size and sample variability.
(i) “”Student information system.”” The student information system is the comprehensive management information system maintained by the Department containing student data reported by school districts in accordance with Fl. Admin. Code R. 6A-1.0014, as incorporated in paragraph (2)(g) of this rule.
(j) “”Value-added model”” or “”VAM.”” A value-added model is a statistical model used for the purpose of determining an individual teacher’s contribution to student learning growth.
(3) Formulas for measuring student learning growth.
(a) The English Language Arts and Mathematics value-added models.
1. The formula for measuring student learning growth using student English Language Arts and Mathematics results approved by the commissioner is a covariate adjustment value-added model.
The value-added model statistically establishes the expected learning growth for each student, called an expected score. When a student’s actual performance differs from these expectations, a portion of that difference is attributed to the teacher’s and a portion is attributed to the school’s influence. Together, this information is used to compute a teacher’s value-added score. School value-added scores are the average of the teacher value-added scores within the school, and are provided to districts that choose to use them for the performance of students component of administrator evaluations required under Section 1012.34(3)(a)1., F.S.
2. The formula for the model. A full technical description of the data sources, formula, covariates, and methodology for calculating VAM scores is provided in the publication, “”Florida VAM Methodology”” (Effective August 2015) (http://www.flrules.org/Gateway/reference.asp?No=Ref-05724), which is incorporated herein by reference. A copy of “”Florida VAM Methodology”” may be obtained from the Florida Department of Education, 325 West Gaines Street, Room 544, Tallahassee, FL 32399-0400.
3. The covariates included in the value-added model approved by the commissioner are:
a. The number of subject-relevant courses in which a student is enrolled.
b. At least one (1) and up to two (2) prior years of achievement scores on the applicable statewide, standardized assessment for each student.
c. A student’s disabilities. The disabilities used within the model are limited to language impaired; deaf or hard of hearing; visually impaired; emotional/behavioral disabilities; specific learning disability; dual sensory impaired; autism spectrum disorder; traumatic brain injured; other health impaired; and other intellectual disability.
d. A student’s English Language Learner (ELL) status. This covariate is used to control for effects related to whether a student is an English language learner and has been receiving English for Speakers of Other Languages (ESOL) services for less than two (2) years; at least two (2) years but less than four (4) years; at least four (4) years but less than six (6) years; or six (6) years or longer.
e. A student’s gifted status.
f. Student attendance.
g. Student mobility. This covariate is used to control for effects related to changing schools during the school year.
h. Difference from modal age in grade. This covariate is used to control for effects related to differences in a student’s age from the most common age for students enrolled in the same grade across the state.
i. Class size. This covariate is used to control for effects related to the number of students in a class.
j. Homogeneity of students’ entering test scores in the class. This covariate is used to control for the variation in student proficiency within a classroom at the beginning of the year.
4. The formula produces a value-added score for a teacher. For English Language Arts and Mathematics, this value-added score consists of two (2) parts:
a. The teacher effect. The teacher effect is an estimate of a teacher’s contributions to student achievement as measured by scores on statewide, standardized assessments. It is based on the difference between expected scores and actual scores for a teacher’s students relative to other teachers in the school, among students assessed in the same subject at the same grade level during the same year.
b. The school component. The school component is an estimate of the part of a student’s performance that is common to students within a school. It is based on the difference between expected scores and actual scores for the school’s students relative to other schools in the state, among students assessed in the same subject at the same grade level during the same year. It represents school-level factors influencing performance of all students in a school among students assessed in the same subject at the same grade level during the same year. Fifty (50) percent of the school component shall be added to the teacher effect to create the teacher’s value-added score.
(b) The Algebra I value-added models.
1. The formula for measuring student learning growth using student results from the statewide, standardized end-of-course assessment in Algebra I pursuant to Florida Statutes § 1008.22, approved by the commissioner is a covariate adjustment value-added model.
The value-added model statistically establishes the expected learning growth for each student, called an expected score. When a student’s actual performance differs from these expectations, a portion of that difference is attributed to the teacher’s and a portion is attributed to the school’s influence. Together, this information is used to compute a teacher’s value-added score. School value-added scores are the average of the teacher value-added scores within the school, and are provided for districts that choose to use them for the performance of students component of administrator evaluations required under Section 1012.34(3)(a)1., F.S.
2. The formula for the model. A full technical description of the data sources, formula, covariates, and methodology for calculating VAM scores is provided in the publication, Florida VAM Methodology.
3. The covariates included in the Algebra I value-added model approved by the commissioner include those listed under sub-subparagraphs (3)(a)3.a.-j., as well as the following:
a. Mean prior test score. Mean prior test score is the average of the most recent prior score on the statewide, standardized assessment in Mathematics for all students within the class.
b. Percent of students in the class who are reported in the student information system as Gifted.
c. Percent at modal age in grade. Percent at modal age in grade is the percentage of students in the class whose age on September 1 of the school year is the same as the modal age of all students in the same grade.
4. The formula produces a value-added score for a teacher. For Algebra I, the score is the teacher effect. The teacher effect is an estimate of a teacher’s contributions to student achievement as measured by scores on statewide, standardized assessments. It is based on the difference between expected scores and actual scores for a teacher’s students relative to other teachers in the school, among students assessed in the same subject at the same grade level during the same year.
(4) Data Collected and Reported for VAM.
(a) Data collection and reporting procedures for purposes of VAM calculations shall be as provided in the publication, Florida VAM Methodology. As set forth in this publication in more detail, data from the Student Information System and Staff Information System obtained from Surveys 2 and 3 are used in VAM calculations. School districts submit Survey 2 and 3 data to the Department’s Student Information System and Staff Information System pursuant to Fl. Admin. Code R. 6A-1.0014 (Comprehensive Management Information System) and Fl. Admin. Code R. 6A-1.0451 (Florida Education Finance Program Student Membership Surveys) (http://www.flrules.org/Gateway/reference.asp?No=Ref-05725). These rules are incorporated by reference herein, and a copy of the rules may be obtained from the Florida Department of Education, 325 West Gaines Street, Room 544, Tallahassee, FL 32399-0400.
(b) Results provided to districts shall include the following information for each statewide, standardized assessment for which a formula has been adopted:
1. A value-added score for each teacher and administrator based on the statewide, standardized assessment associated with the course(s) that the teacher taught during the current year or the school the administrator was assigned to during the current year. This score shall be reported for each grade level and subject area covered by the statewide assessment.
2. Three-year aggregate value-added scores for each teacher, which includes data for the teacher from the current school year and each of the two (2) prior years for which data are available, for a total of at least one (1) and up to three (3) years of data for the teacher, as follows:
a. Three-year aggregate English Language Arts score. A combination of all value-added results for the teacher from all grades and courses associated with the statewide, standardized assessments in English Language Arts.
b. Three-year aggregate Mathematics score. A combination of all value-added results for the teacher from all grades and courses associated with the statewide, standardized assessments in Mathematics.
c. Three-year aggregate combined score. A combination of all value-added results associated with the statewide, standardized assessments in English Language Arts and Mathematics.
d. The aggregate score shall be calculated by standardizing the value-added scores by converting them to a proportion of a year’s average growth within the grade and subject for the year, and combining them across all grades and subjects for as many of the last three (3) years as data are available. For districts choosing to use school score(s) for administrator evaluations, districts may elect to combine multiple one (1) year, two (2) year, and three (3) year aggregate scores that reflect the school(s) to which the administrator was assigned and the years to which they were assigned to those school(s) during the period.
3. The standard error for each value-added score.
4. For each value-added score that contains a school component, the school component and teacher component reported separately.
(5) Classifying and Interpreting Scores.
(a) Data elements used by the department to classify and interpret scores are as follows:
1. The statewide average year’s growth for students in each grade and subject. For each student learning growth formula, an average year’s growth for students across the state on the statewide assessment is calculated, and once standardized, uses a threshold of zero (0) to establish performance expectations. A score of zero (0) indicates that a teacher’s students scored no higher or lower, on average, than expected.
2. The educator’s value-added score.
3. The confidence interval. A confidence interval is computed using the standard error associated with the educator’s value-added score.
(b) Ratings. Districts may use the rating provided by the department for the performance of students criterion in performance evaluations under Florida Statutes § 1012.34, for classroom teachers of courses associated with statewide, standardized assessments and administrators.
1. Ratings for Florida’s value-added models. The Department provides ratings for grade-specific and aggregate scores from the English Language Arts, Mathematics, and Algebra I value-added models.
2. The ratings for the English Language Arts, Mathematics, and Algebra I value-added models are as follows:
a. Highly Effective. A highly effective rating is demonstrated by a value-added score of greater than zero (0), where all of the scores contained within the associated 95-percent confidence interval also lie above zero (0).
b. Effective. An effective rating is demonstrated by the following:
(I) A value-added score of zero (0);
(II) A value-added score of greater than zero (0), where some portion of the range of scores associated with a 95-percent confidence interval lies at or below zero (0); or
(III) A value-added score of less than zero (0), where some portion of the range of scores associated with both the 68-percent and the 95-percent confidence interval lies at or above zero (0).
c. Needs Improvement, or Developing if the teacher has been teaching for fewer than three (3) years. A needs improvement or developing rating is demonstrated by a value-added score that is less than zero (0), where the entire 68-percent confidence interval falls below zero (0), but where a portion of the 95-percent confidence interval lies above zero (0).
d. Unsatisfactory. An unsatisfactory rating is demonstrated by a value-added score of less than zero (0), where all of the scores contained within the 95-percent confidence interval also lie below zero (0).
Rulemaking Authority 1012.34 FS. Law Implemented Florida Statutes § 1012.34. History-New 9-9-15, Amended 4-30-18, 3-15-22.