Community of Practice Evaluation Strategy

This post is one in a series sharing the case study of the development of a Community of Practice for Maintenance and Engineering teams at Coca-Cola Amatil.

As discussed in the case study post on context, this initiative is expected to contribute to reducing equipment down-time and decreasing risk associated with concentration of deep process knowledge in a small number of long-tenured Engineers.  A number of Key Performance Indicators (KPIs) were identified up-front to evaluate the value created by this community.  This was informed in part by the conceptual framework of value creation from online interactions from Etienne Wenger, Beverly Traynor and Maarten De Laat.  See Jane Bozarth’s article for an introduction to the value creation cycles in the framework.

Value creation cycles

KPIs and Measures

MaintEng COP KPIs

Measurement Points & Methods

Three main measurement points have been identified:

  • Prior to community launch – to establish a baseline
  • Immediately following the Work Connect & Learn program (guided social learning program to support the development of skills and behaviours to participate in the community)
  • Six months after completion of the Work Connect & Learn (WCL) program

Monitoring of some of the measures will occur on a monthly basis during this six month period to help inform community management.

Data and feedback will be gathered using the following methods:

  • Surveys of all maintenance and engineering team members at the three main measurement points
  • Monitoring of activity on online community spaces in SharePoint
  • Interviews / discussions with Maintenance & Engineering Managers and Capability Managers (who are providing on the ground supporting to community members to participate)
  • Focus groups with community members immediately following and six months after the WCL program
  • Capture of value creation stories on an ongoing basis in the six months after the WCL program

Additional Thoughts on Value Creation Cycles

The careful reader may have noticed that indicators of Reframing Value have not been explicitly included in the KPIs.  While we have not explicitly set objectives for value at this level, it will be surprising if this does not follow success in the other value creation cycles.  Reframing Value will be identified through value creation stories.

The value creation framework has recently been expanded to include strategic value and enabling value.  We’ve not yet considered whether / how to include these in our evaluation strategy.

1 Comment

Staying up to date with Google Alerts via Feedly

Last November I set up some Google alerts in order to receive automated updates on the latest internet search results on topics that I’m interested in.  It distills the results from the first 20 or so pages of its search engine results and sends you an alert.  Rather than receive results by email, I set my alerts up as an RSS Feed into Feedly by following the instructions at How to Curate Content With Feedly and Google Alerts .  (For those unfamiliar with RSS Feeds, refer to this article for a straightforward explanation.)

Google Alerts

A scan of the alerts that I set up shows that I’ve used alternative terms with minor variations to increase the range of search results.  You’ll also notice the use of quotation marks around phrases to ensure that results only include items that use the full term,not the indivudal words spread out throughout an article. (Here’s more tips on setting up Google alerts.)

Automating searching allows me to be fed new articles and information on topics I’m interested in.  So long as I am reading on Feedly a couple of times a week it is a reliable way to be pushed new content without having to make the effort to do manual searches.

Reading Alert results helps me to stay abreast of industry trends and developments.  It supplements the content that the “human filters” in my online Personal Learning Network feed me through Twitter, LinkedIn and the blogs that I follow. It also allows me to increase my value and contribution to my networks by sharing content that may not already be in circulaton in those networks.

Google Alerts have been a valuable addition to my Personal Knowledge Mangement toolkit.

4 Comments

702010 Forum Value Creation Story

I came across the idea of using value creation stories to assess the value of online interactions in an article by Jane Bozarth.  She used a conceptual framework from Etienne Wenger, Beverly Traynor and Maarten De Laat to write the story of how her online interactions create value.  Their framework thinks about value in terms of five different cycles, which Jane summarises clearly in a diagram in her article.

Value creation cycles

Value creation cycles

As Jane urges, I have read the full text of this framework, and have used their guiding questions to write a value creation story about my participation in the 702010 Forum.  This is my practice run in preparation for gathering value creation stories to evaluate communities that I am helping develop in my organisation. I’ve written this post by responding directly to the guiding questions relating to each cycle of value creation.  However, I would document value creation stories within my organisation by writing or recording (audio or video) a more natural narrative.

702010Forum banner

The 702010 Forum is a community of practice for learning and performance professionals applying the 702010 framework. Membership is subscription based, with some resources and webinars publicly available.  A toolkit contains resources to support application of 702010, and members can participate in a range of events and an online discussion forum.

1. What meaningful activities did you participate in? (Cycle 1 – Immediate Value)

I have participated in a variety of asynchronous and synchronous activities including:

– attending webinars or viewing recordings, especially case studies by members
– delivering two case studies on webinars
– attending face-to-face events
– initiating and replying to discussions in the forum – and extending this in one case to email and phone discussions on a solution shared in the forum (more below)
– participating in a pilot of the Forum’s 702010 Practitioner Certification
– sharing and applying resources from the Toolkit with others in my organisation to assess status and develop improvements

Toolkit snapshot - showing entries I've ticked off as read

Toolkit snapshot – showing entries I’ve ticked off as read

2. What specific insights did you gain?  What access to useful information or material? (Cycle 2 – Potential Value)

702010Forum Webinar RoleWithin a few months of joining the Forum I realised that while my organisation had adopted the 702010 framework a number of years ago, we had narrowly interpreted it.  We were deploying blended learning where we used activities based on learning from experience (70) and others (20) within formal structured programs.  However, we were not purposefully enabling people to learn as they worked or building ongoing social learning capability. From a webinar on the changing role of the learning function I saw that the skills of our capability team needed to be broadened. I gained an understanding and language to talk to key stakeholders about the opportunity to impact organisational performance more effectively if we added performance consulting, performance support, and social learning to our toolkit.

3. How did this influence your practice?  What did it enable that would not have happened otherwise? (Cycle 3 – Applied Value)

These activities and insights enabled me to have different discussions about 702010, our learning strategy, and internal Capability skills, particularly with senior managers and our Capability team.  The most significant shift it enabled was an update to our Capability strategy in April 2014 to include ‘Continuous Workplace Learning’ as an element.  This broadened the remit of our Capability team and created the space for us to get strategic with social learning.

We revamped our Governance Board by applying the “Toolkit for Establishing a Learning Governance Board.”  Consequently we get better value out of our quarterly meetings by focussing on alignment with our business strategy.

Tonkin & Taylor Knowledge Shots webinar

Tonkin & Taylor Knowledge Shots webinar

A specific initiative accelerated by a case study in a Forum webinar was setting up a knowledge sharing site on SharePoint.  When a senior manager gave me 1 day to prepare a prototype of a community hub on SharePoint I recalled a webinar where Tonkin & Taylor demonstrated their Knowledge Shots solution.  I incorporated this into our prototype, and went on to build a variant of this for my organisation.  You can see what this looks like in the guided tour of a community hub now set up on SharePoint.  (Big thank you to Tammy Waite and Mark Thomas from Tonkin & Taylor, Forum members, for the support they provided by email and phone.)

 

4a. What difference did it make to your performance?  How did this contribute to your personal/professional development? (Cycle 4 – Realized Value)

As per the examples given above, application of ideas, tools and solutions from the 702010 Forum has enabled me to perform and contribute to my organisation in ways I may not otherwise have been able to.

The 702010 Forum was one of the first Learning & Development communities that I’ve participated in.  During 2014 I got active on Twitter and started this blog.  This helped me to create a Personal Learning Network (PLN) which encompasses many other communities (e.g. Third Place, OzLearn, PKMChat).  Collectively my participation in a number of communities and interaction with my PLN has transformed and accelerated my professional development.

My first webinar delivery

My first webinar delivery

One way that the Forum has supported my development has been by increasing my confidence to work out loud publicly in order to learn and improve.  In December 2013 I delivered a case study in a Forum webinar.  I found the reflection and learning from delivering this webinar valuable, and saw that others could benefit from me working out loud.  This was a catalyst – it gave me the motivation and confidence to continue working out loud by blogging and speaking at conferences.

4b.  How did this contribute to the goal of the organisation?  Qualitatively?  Quantitatively? (Cycle 4 – Realized Value)

Realised value for my organisation is unfolding.  It’s also difficult (and unnecessary) to unravel the influence of the 702010 Forum on my organisation’s performance versus that of other communities and networks I participate in.  Our Capability strategy is better aligned to our overall business strategy as a result of applying ideas and tools from the 702010 Forum.  We are using a broader range of Capability approaches and activities to achieve our goals.  I think it will be another 6-9 months before we are clear on the outcome of these activities.

5. Has this changed your or some other stakeholder’s understanding of what matters? (Cycle 5 – Reframing Value)

An emphatic YES to this question.  It’s shifted the perspective of two very important stakeholder groups, senior managers and the Capability community, about the importance of the 70 and 20 elements of the framework and the range of approaches we can use to build Capability.  We’ve reframed our Capability strategy, launched communities of practice and embraced performance consulting.

Participate, Participate, Participate

I shall close by encouraging everyone reading this to reflect on the communities and networks they are part of and consider their current level of participation.  The more you participate, interact with others, apply ideas from these groups and share back what happened, the more value you create for yourself, your organisation and other community members.  So, what are you waiting for?  Get in there and participate.

I’d love to hear about the value that others have found in participating in communities and networks – you can leave your thoughts below or pingback to your own blog posts.

 

 

 

8 Comments

A Few of My Favourite Blogs

feedly logo

I subscribe to a range of blogs using Feedly as my RSS Reader.  This makes it easier to keep up to date with industry blogs and reduces email clutter.  I currently have 488 unread Google alerts and 211 unread blog posts in Feedly. I could do better with regularly checking and reading my subscribed feeds.  I tend to Feedly categoriesfocus on a small number of my favourite blogs.  Here I reflect on what I enjoy about three of these.

Harold Jarche

Over 10 years of blogging Harold Jarche has published 2,650 posts. Two themes I enjoy are Personal Knowledge Management (PKM) and networked working and learning .  He continually evolves his thinking on topics, reusing, updating and refining content.  I like his approach of writing primarily for himself.  If others find value in his work then that’s a bonus.  I use this as a model for my own writing.  It reduces the pressure , and helps me to focus on learning and improving my practices.  While not quite stream of consciousness, I can see his thinking and work developing over time.   He has compiled his “best posts” into two e-books.  I’ve read the first, Seeking Perpetual Beta, which he Perpetual Betadescribes as “a cohesive narrative that covers learning, working, and managing in the emerging network era”.  While he writes clearly I sometimes feel too trapped by my paradigms to see how to apply his vision of the future of work in my world.  His thinking stretches me and motivates me to question the status quo.  His practical guidance on personal and organisational knowledge management is valuable.  I completed his PKM in 40 days program in 2014.  This gave me skills to filter relevant information and make sense of it.  His model of “How Enterprise Social Networks (ESNs) add Value” is a useful framework.  It has helped me speak with people in my business about why and how we can better use our ESN.  I look to Harold as a pathfinder, helping me to find different ways of working and learning.

Helen Blunden

Helen and me working together - after I got to know her through her blog

Helen (on right) and me working together – after I got to know her through her blog

Helen Blunden writes on her business blog, Activate Learning Solutions.  While not as prolific as Harold she does write often – 13 posts in first 2 months of 2015.  Helen writes about modern learning approaches, her networking activities and working experiences.  I most enjoy the case studies where she describes programs she has developed and how she worked.  Her social on-boarding case study is a good example.  The open, detailed way she writes gave me a good sense of who she is and her professional approach.  Helen is inquisitive, interested in others, seeks to understand the business environment and people, designs practical solutions to improve performance and results, and has an eye for detail. After several months of reading Helen’s blog and connecting with her online I met her briefly at a conference in mid 2014.  Face-to-face she was consistent with her online persona.  I felt that I knew her well from our online interaction and portfolio on her blog.  I did not hesitate to engage her to help develop a Community of Practice.

Sacha Chua

Sacha chua

Finally, the blog I get the most pure pleasure in reading is Sacha Chua’s Living An Awesome Life.  Sacha is in the midst of a five year semi-retirement experiment that she started in her late 20s.  I admire her courage and resourcefulness in making this happen.  The way she thinks is fascinating.  She is an astute observer who asks interesting questions, breaks down a topic into smaller pieces to analyse and develop  insights, and provides helpful visual summaries (sketchnotes).  Her writing is simultaneously intensely personal and broadly relevant – as exemplified by her recent post on common goals.  Her blog is a place to think and learn.  She posts almost daily, and she has written over 7,000 posts in 14 years.  Her motivation, originality, openness and willingness to share are inspiring.  She makes me want to live a better quality self-directed life.

You can imagine my delight when I recently saw a video of Sacha and Harold discussing blogging and PKM on YouTube.  The two take very different approaches to developing their thinking and managing their blogs.

What are your favourite blogs and why? 

5 Comments

Community of Practice Case Study

In September 2014 we decided to get strategic with social learning in Coca-Cola Amatil’s Supply Chain and establish a Community of Practice for our Maintenance and Engineering team members across Australia and New Zealand.  Helen Blunden of Activate Learning was engaged to help with this initiative.  Helen and I are Working Out Loud about the development of this Community.

This post provides links to all our posts which collectively form a case study that we’ve developed as we worked.  I’ve arranged them in a  logical sequence for you to read if you wanted to follow the case study in rough order in which things happened (although there is a lot iteration).  I’ll keep adding to it as we post more on this case.

1.  Context - Michelle’s post about the background to the decision to get strategic with social learning and establish the Community of Practice

2. Strategy and Analysis Phase – Helen’s post about how the need was analysed and strategy for the Community of Practice developed

3. Work, Connect and Learn Program – Helen’s post about the guided social learning program we decided to develop to help community members learn skills and behaviours required to participate in online community activities

4. Development of the Work, Connect and Learn Program – Helen’s post about how the program was developed

5. The online community hub – Michelle’s guided tour of the community infrastructure set up using standard SharePoint 2013 functionality, integrated with Microsoft Lync and OneNote

6. Change Management approach – Michelle’s post (to be written) about the approach taken to change management before Community launch

7. Evaluation strategy – Michelle’s post about the business objectives and evaluation strategy for the Community

8. Pre-Program Survey – Michelle’s post (to be written) about what we learned about our community members through a survey before we started the Work, Connect and Learn program

9. Reflections on Module 0 of Work Connect and Learn – Helen’s video log reflection before and after delivering webinars for Module 0 Learn How to Learn Online

10. Reflections of the Work Connect and Learn program – Helen’s post about lessons she learned during delivery of the Work Connect and Learn program

, ,

3 Comments

Community Hub on SharePoint 2013

To support the launch of Coca-Cola Amatil’s Maintenance and Engineering Community of Practice we’ve built an online Community Hub in SharePoint 2013.  I invite you to take a video tour of this Hub.  As the tour is 15 minutes I have also provided a brief written description of the spaces set up on the Hub for Community members to connect, share and collaborate. (Note – I’ve recorded this video using Microsoft Lync.)

Key Hub Features

The Hub sits on our Maintenance Engineering SharePoint site, and is basically a dashboard with links to a range of spaces and resources on both SharePoint and the internet.  In the video I walk through the spaces we’ve set up on SharePoint for Community members to interact online, connecting, sharing, solving problems and improving practices together.  All of these spaces are accessible from any mobile device or computer using an organisational login.  The mobile access is particularly important for maintenance tradespeople who spend most of their time in the production environment, away from desks.

CCA Community Hub

The CCA Links on the hub are:

Discussion Forum – standard discussion board with ability to create new discussions, reply and like.  The board can be sorted and filtered in a number of ways, and search is available on the site which will include discussions in results.  A useful function I don’t discuss in the video is that an individual can quickly set up an alert to receive notifications of changes on the discussion forum via email (immediately, daily or weekly).  This is helpful to stimulate those who aren’t working on SharePoint regularly to participate in forum activity.

Shared Notebook – a Microsoft OneNote notebook set up on the Maintenance and Engineering site.  OneNote is a very easy to use, flexible collaborative tool.  We have set up a section for use during the Work Connect and Learn program, and there is currently another section being used for Maintenance Managers meetings.  We expect to see use of OneNote increase as Community members become familiar with it.

Contact Directory - a directory with expandable sections containing SharePoint profile key data.  A person can be added to multiple sections reflecting the different groups to which they belong (e.g. Maintenance Managers, members of a physical site team, system Key Users).  From each person’s directory entry you see whether they are online, start a Lync IM chat or call, create an email, access their contact details, or open their SharePoint personal page.

SharePoint directory

 Example of Contact Directory popup with interfaces to Lync and SharePoint personal page

Supply Chain Knowledge Bites – a separate SharePoint site where anyone can share a short ‘bite’ of knowledge or ‘how to’ information.  Documents and multi-media files can be included in a Bite, along with links to internet resources.  People are often unsure about where or how to share their expertise, so we’ve set up this space to provide a common way of sharing and accessing documented know-how.  The second half of the video explores Knowledge Bites in more detail.

Community members are learning how to use these spaces along with other collaborative tools (notably Microsoft Lync) through our Work Connect and Learn program.  Program activities are being conducted in these normal working spaces so that people get used to using these spaces during the program, and continue to use them afterwards.

How the Hub was Built

Work on the hub commenced with a PowerPoint prototype which a colleague, Justine Jardine, developed with me in just one working day at the request of a keen senior manager who wanted to champion the solution at an annual planning meeting.  We were able to respond so quickly in part because we had seen a demonstration of a knowledge sharing space set up on SharePoint 2013 by a New Zealand engineering firm, Tonkin & Taylor, in a 702010 Forum webinar. We included additional spaces and resources in the prototype, all of which we modelled on existing internal SharePoint sites.  So, our prototype was low risk as we had working examples of all functionality included.

While our internal IT department provisions SharePoint sites using a standard organisational template, they do not build any functionality on sites.  They also do not allow custom development, which ended up precluding some of the prototype features as Tonkin & Taylor had done custom development.  Further, we were unable to use SharePoint ‘Community features’ as IT is still trialling these.

So, I developed the Hub using standard SharePoint 2013 functionality.  While I knew how to build a dashboard, almost everything else I had to learn how to construct.  I searched the internet for information and ‘how to’ articles and videos, with Helen Blunden (who was developing Work, Connect and Learn) researching SharePoint 2013 features alongside me.  There is a LOT of freely available information about how to set things up in SharePoint 2013.  I also got tips from our SharePoint SME in IT (he had time for quick questions), and appreciated some discussion with contacts at Tonkin & Taylor about their Knowledge Shots solution. Of course, the other way I learned was to experiment – to build things and see what they looked like, then adjust.

Several members of our internal Capability Community tested and reviewed the build as Hub components were developed.  Often their excellent suggestions could not be implemented with standard SharePoint 2013 functionality, but did push me to figure out different ways to do things as we continued iterative development.  The experience of building this Hub has made me more resourceful as a self-directed learner, and more likely to figure things out for myself than wonder why IT hadn’t “trained me” on using SharePoint.

 Updates

At some point I’ll post an update about how Community members responded to the Hub and what adjustments or additional features we introduce.  If you have any suggestions or feedback please leave a reply.

, ,

7 Comments

Sharing Resources and Links with my Team

I enjoy it when someone shares a useful tip, tool or resource with me.  If they’ve thought carefully about my interests, know what’s relevant to me, and offer it to me in a “place” that’s convenient to me, then it’s a great gift from them.  This helps me to think about what and when to share with my team – to do it in a way that my sharing is a contribution and not noise or a burden.  I am also very conscious that I am role modelling new behaviours within my organisation, so want to help people to see the potential value in sharing by doing it judiciously and well.

The internal group with whom I work most closely are the Supply Chain Capability Community. The Community includes 12 people in a range of roles who collectively develop, plan and implement initiatives to improve business performance through the technical knowledge and skills of our people.  I share links and resources with this group in a variety of ways:

1) Verbally or in email with an individual or small group – a resource relevant to the specific topic or context e.g. an article on why measuring performance impact is more important than ROI shared directly with a team member who was designing a new evaluation approach for a learning program.

2) Scotty tweetPosting a link on Twitter and @mentioning specific team members – this works where the team members I want to share with use Twitter (4 out of the 12).  I thought I had done this several times, however when I did an advanced search on Twitter for examples I could find only one where I shared an article on leadership styles in different cultures with a team member who does a lot of work in Indonesia.

SharePoint post 702010 share

3) Posting a link or message about a resource on SharePoint newsfeed with a comment about why I am sharing this link – this is useful where the item is of potential value to a larger range of people in the group.  In the example I highlight a case study on the 702010 Forum in which we have organisational membership.  I don’t share resources in this way very often (despite there being a lot of relevant resources I could share) and have just resolved to add this to my daily sharing habits.  The other thing I could improve is to limit number of characters in post so people don’t need to click on ‘Show more’ to see the whole post, especially if links are at the end of the post.

4) Knowledge sharing sessions with the group during our SharePoint blogfortnightly Community teleconferences  – we’ve replaced status updates with knowledge sharing and learning discussions in these regular catchup sessions.  Format is presentation followed by discussion.  Presentations are most commonly on a topic (e.g. Gamification through badges), a work practice (e.g. how we can increase manager support to learners), or report back on ideas from an external event such as a course or a conference. I’ve also written a blog post on a topic with linked resources and asked people to read and reply to questions before the session (example shown is for a discussion on Working Out Loud). Participation in online discussion has been low and the group interacts far better in synchronous discussion than asynchronous (I hope that the Work, Connect and Learn program will help increase online interaction).  This is a valuable forum for our team to Work and Learn Out Loud together, and we shall continue to use and fine tune it.

5) Diigo – I set up a Diigo account for the team to use to curate and share online resources.  I am the only one who curates on a regular basis; however there are several team members who are comfortable using Diigo for joint research to meet a specific need.  Below is an example of curation of research into modern approaches to learning design.

diigo future design

6) “Learning Links” blog posts – I have started to share short collections of resources on a SharePoint Learning Linksspecific topic relevant to the group (e.g Social learning) on our SharePoint blog.  I write a short commentary about each resource and any overall themes.  As I bookmark items to Diigo I tag them with “LearningLinks” if I think they my be worth including in a post at a later date.  My intent had been to do a weekly Links post, but I’ve been erratic so have diarised this.  I shall also start posting these collections on my internet blog in case they are useful to others outside my organisation.

Diigo is at the heart of my resource sharing practices, allowing me to bookmark and tag links that I can re-use and share in a range of contexts and ways.  Being able to store links in a library that I can access anywhere I have an Internet connection means that I can share good quality resources at the right moment and with the right people to create value rather than generate noise.

1 Comment

Follow

Get every new post delivered to your Inbox.

Join 806 other followers

%d bloggers like this: