Case Study

Lightspeed Systems® and Digital Promise have joined forces to provide participating Digital Promise League of Innovative School Districts a better way to track and monitor student-level digital engagement: the use of Lightspeed Digital Insight™. This award-winning data analytics tool enables schools to analyze the efficacy of their technology investments and identify those students at risk of falling through the cracks—all in real time Seven districts ranging in various sizes and locality participated in the project, including Virginia’s Chesapeake City Public Schools (CPS). CPS is a Lightspeed Systems customer and a member of the Digital Promise League of Innovative Schools. CPS agreed to include their aggregated, identified application usage and engagement data in the pilot study to help provide insight into the usage and engagement patterns of students in digital learning environments.

Identifying The Paid Tools Used By Students In Specific Courses

CPS’s initial data analytics objective was to build an established data set from which they could begin making analysis and inform decision-making. “I think it is really important to have longitudinal data,” said Jeff Faust, CPS’s Chief Technology Innovation Officer. “To base a decision on what software is used in a month or two is not fair because there are a lot of tools that are not used in the first two months of school. Those same tools might become really important in the second semester. So, with Lightspeed Digital Insight, we wanted to get its agents pushed out to our student and staff devices as fast as possible and start collecting data. That was our original focus, but it is amazing how quickly our usage of the tool has become much bigger than that.”

With Lightspeed Digital Insight, we wanted to get its agents pushed out to our student and staff devices as fast as possible and start collecting data. That was our original focus, but it is amazing how quickly our usage of the tool has become much bigger than that.

CPS's Expanded Vision for the Application and Usage of Data Analytics in Education

Executing An Effective Change Management Strategy

As was the case for school districts across the country, video conferencing became an essential tool for delivering remote instruction for CPS. While the district’s leaders were very supportive of live instruction being delivered in this collaborative format, they did have security concerns around one video conferencing tool and are shifting users to a different platform in 2021–2022. To ease the transition and encourage early adoption of the new platform, the leadership team created dashboards within the customizable platform for each school that showed the usage comparisons for both tools. “Now that we have the data, we can share reports at our superintendent meetings as well as at our elementary, middle, and high school principal meetings that highlight who is doing a great job at embracing the new platform,” said Faust. “It has been a fun contest for them, and it has been an effective change management strategy because we have not put a lot of pressure on them to make the immediate switch. Instead, we are letting the data and application usage trends drive the change.

Now that we have the data, we can share reports at our superintendent meetings as well as at our elementary, middle, and high school principal meetings that highlight who is doing a great job at embracing the new platform.

Informing Licensing And Budget Decisions

CPS’s district leaders are looking forward to having a year’s worth of data to help inform their licensing renewal and purchasing strategies. “I have to review every software license that comes across my desk,” said Faust. “Now, every time I get a request for software, I enter the license into Lightspeed Digital Insight. Once we have a full year of data, we will have a library of what we are missing right now, which is who has what software, who is paying for that software, and so on.” The district intends to use that information to identify opportunities such as buying districtwide licenses, as opposed to individual site licenses, or scaling down licensing to save money where it makes financial sense to do so.

“We want to move away from the practice of continuing to buy certain programs or tools because they have a strong legacy in education,” said Faust. “Instead, we are going to be asking more critical questions around our renewals and licensing, such as, ‘What’s the adoption like? What’s the effectiveness of the product? Is it helping our students and teachers achieve their goals?’ And those are tough questions to answer if you only have subjective and circumstantial information. Lightspeed Digital Insight provides us objective data we can use to inform and answer these questions.”

Faust has labeled this methodology as the cow path model, which essentially means you pave the paths where walkways already exist instead of trying to design new, unique paths because it pleases you. Inevitably, people will circumvent your pathway and still walk over the grass. “I want to renew and purchase software that our staff is using or will use, and that is effective,” said Faust. “And I want to provide the training they need to use it. But I do not want to provide training and force our teachers to use a platform or tool just because we bought it.”

Tracking Student Engagement And Attendance

Measuring student engagement and attendance became an entirely new challenge during the 2020–2021 school year. In fall 2020, CPS implemented a hybrid learning model where students could choose to either attend class in-person or virtually. The district’s educators were also teaching in a dual-roster model, meaning they were delivering instruction to both students in the classroom and students at home simultaneously. What made the dual-roster model particularly difficult for teachers was the unpredictable nature in terms of student attendance. For example, a teacher may have prepared their lessons with the expectation that they would have 12 students in the classroom and four kids at home. In reality, they would unexpectedly have four kids show up in the classroom and 12 students attend virtually.

The CPS team began using Lightspeed Digital Insight engagement data as a secondary metric to monitor attendance and off-campus usage. “We had a lot of students who signed up for in-person instruction, but they were not showing up,” said Faust. “We were able to look at our analytics and validate that the students were joining virtually. We also began using the data to track the number of days students actually showed up in-person and then made adjustments based on that data. For example, we could look at the data and see that only 20% of students were on a particular campus on any given day. Based on that information, the principal could opt to bring more students back on campus because they weren’t at capacity.”

We were able to look at our analytics and validate that the students were joining virtually. We could look at the data and see that only 20% of students were on a particular campus on any given day. Based on that information, the principal could opt to bring more students back on campus because they weren't at capacity.

Uncovering Unlicensed Apps

CPS created dashboards for principals at each school to help them implement change management strategies, track student attendance, and monitor the apps being used by their students. Each principal has access to a dashboard that has the top 10 licensed and approved apps being used and the 10 top unlicensed apps being used.

CPS leaders hope that visibility into unlicensed app usage will enable district and school leaders to draw some correlations between application usage and student performance and inform future decision-making and instructional strategies. For example, Lightspeed’s data shows that some classes are using a popular math program, but it is not licensed by CPS. “This particular tool is fun and engaging, but the question is whether it is actually effective or not,” said Faust. “That remains a bit of a gray area. Using Lightspeed Digital Insight, we now know which schools are using that tool. Correlation is not causation, but what if those are the same schools where the students are performing the best on their end-of-the-year assessments? If that’s the case, maybe we need to have a different conversation around that math tool. Tracking these kinds of ‘sneaky’ applications being used that we did not know about has been valuable for us.

Fostering Data Privacy Conversations

CPS is leveraging the Lightspeed Digital Insight privacy alerts and application usage data to identify schools that have adopted and are actively using tools that do not meet the district’s privacy standards. Equipped with this knowledge, district leaders are able to notify school leaders of this usage and provide them with recommendations for other platforms that are very similar but comply with the district’s student data privacy policies.

Creating A Data Advisory Committee

To foster more conversations around data and how to put their analytics into action, CPS formed a data advisory committee. These committee members are examining their adoption and software acquisition procedures in relation to their application usage and engagement data and addressing questions, such as the following:

• What is the software that we are using?
• How do we know if these platforms are working?
• Did we know this tool or platform is being used?

Committee members are also working with the district’s instructional leaders to perform an audit of the free applications being used at CPS. They are also helping instructional leaders facilitate conversations regarding which free applications the district’s students and staff should or should not be using. “You cannot really begin to change a practice until you know what’s happening,” said Faust. “Through this data, we are getting the big picture view we need to establish and implement some effective strategies to change practices. It has also helped transform our culture by redefining what we need to ask and what awareness we need to have.

Key Insights and Next Steps

CPS’s data journey progression is certainly impressive. Moving forward, the district intends to further integrate data analytics to expedite their software approval process and further strengthen their compliance efforts at the educator level.

“We want to make the software acquisition process as painless as possible and take the bureaucracy out of it,” said Faust. “But we also want to make sure it is a good process, so we want to make sure we are asking mportant questions, such as, ‘What is the privacy policy? Did the vendor sign the student privacy pledge? Does the vendor have any standards that they are certified and compliant with?’ Those types of questions have been put at the top of our software approval process because we want our principals and teachers thinking about those things too. And we want them familiar with organizations like IMS Global Learning Consortium and doing some initial research to make sure the software they are requesting is safe. Having access to Lightspeed Digital Insight has really started some healthy conversation’s for us around data literacy and data awareness.