Thanks to funding from the Chan Zuckerberg Initiative, we recently completed a medium-term evaluation of our foundational training course, Scientific Community Engagement Fundamentals (CEF). We’ve just published a report that summarizes the results of this work, which shows impact across three levels of scale – the individual, their community/organization, and the wider STEM ecosystem.
In this blog post, we’ll recap some of the rationale for the report and a high level overview of our findings. Subsequent posts will share more about our user-centered design approach to creating professional training courses, what we learned about the impact of CEF at each level, and how this work will impact our ongoing training offerings.
What we wanted to know
At the heart of this project was a desire to learn more about how participants in CEF apply what they learned in the months or years after completing the course. While we have consistently collected exit survey data, asking questions about immediate impacts and applications of the course themes, this was our first opportunity to follow up with our learners to determine what happened next. Did they actually use the course concepts again? Which frameworks were most useful over time? Did community engagement in their own communities improve? What other impacts have they benefited from, such as ongoing connection to their classmates and maybe even new collaborative opportunities?
What we learned
In brief, our findings indicate that:
- 95% of respondents have applied multiple CEF concepts, frameworks, and activities in their roles to improve overall community strategy (68.6%), improve existing content and/or programming (67.4%), develop new content and/or programming (64%), and build alignment with team members (55.8%) – among other things.
- On an individual level, the vast majority of respondents reported positive socioemotional outcomes such as increased confidence (87%) and connection to peers (90%).
- On a community / organizational level, 74% of respondents reported at least one improvement in member participation within their communities, and 51% reported multiple improvements – as described by deeper engagement across the modes of CSCCE’s Community Participation Model. An even greater percentage (79.1%) selected multiple additional improvements beyond member engagement, such as such as the development of scaffolding and community engagement strategy.
- On an ecosystem-level, 38% of respondents reported multiple improvements in their organizations’ and communities’ relationship to the broader STEM ecosystem, such as improved communication with external stakeholders or communities, greater clarity about their community’s place within the broader STEM ecosystem, and new collaborations with external organizations or communities.
What learners said
We are particularly grateful that many of the respondents to our survey took the extra time to share more about their experiences in their own words. In terms of how the course and ongoing professional network supported their work, they said:
“I was able to articulate and understand how community managers are different from science communicators; some of the skills I’ve learned have now become second nature — i.e. I have extremely organized notes that are scaffolded well for every meeting, I write clear intentions and purposes when drafting out strategy documents, and I’m able to confidently run zoom calls with my community that include an equal mix of socialization and actual work. One of the biggest ways that I’ve grown is recognizing that I have the knowledge, experience, and skills to do my job well!”
“The most impactful part of this course was meeting others in similar positions, and feeling validated in the amount of time, energy and creativity it takes to manage these communities on top of our other day-to-day tasks. This helped me communicate better with my manager and allocate our time resources more effectively.”
Respondents also reported that their members were more engaged, with specific examples illustrating what this looked like:
“We have completely overhauled our community engagement strategy using frameworks, etc. from CEF, standardizing our approach to managing 15 different community roles. We plan to implement that strategy in 2024. But we are already seeing an increase in engagement from changes we have begun implementing.”
We also learned more about the impact of various CSCCE resources and activities, and according to our learners, these are the most important concepts and frameworks they’ve been applying:
- The CSCCE Community Participation Model (CPM) – The CPM describes four modes of member engagement – CONVEY/CONSUME, CONTRIBUTE, COLLABORATE, and CO-CREATE, as well as a fifth “meta” mode, CHAMPION. The model can be used to assess or design programming, with programming in multiple modes ensuring that more of your members can participate in ways that work for them.
- Scaffolding – We define scaffolding as the supportive information, activities, and processes that address barriers to member participation and ensure that all members can access and engage in community programming, and we have an entire page devoted to it on our website. There you’ll find guidebooks, templates, and more to help you create effective scaffolding for your community.
- Conducting a content and programming audit – For more on this, you’ll have to sign up for the next offering of CEF…
Join the fall 2024 cohort of CEF
We generally run CEF four times a year, but with two private cohorts running this summer, your next opportunity to join us at the regular time will be in September. CEF24F will run on Mondays and Thursdays beginning Thursday, 12 September until Thursday, 31 October 2024. The deadline to register is 23 August – but don’t delay too much, the course is already half full!
Join our upcoming Australia/NZ cohort of CEF
Speaking of private cohorts, there are a limited number of spots available for general registration in the cohort of CEF we’re offering in partnership with the Australian Research Data Commons! Live sessions will take place at 7:30–9am AWST / 11:30am – 1pm NZST (find this time in your time zone), and the course will run on Tuesdays and Fridays from 31 May through 19 July. This timing may additionally work for those in Hawaii, Alaska, and the US Pacific time zone.
Work with us
We regularly conduct research for our clients – from program evaluations like this to researching the needs of members in planned or existing communities. If you’re interested in working with us, please reach out to training@www.cscce.org and we’ll set up a call to talk more about your specific needs.