November 15, 2014

Learning Analysis of Social Networks

0 comments

This is my first look into social network analysis for learning.  We are starting with the idea that there is value in understanding how interactions happen during learning regardless of the context.  This sets the stage for digging in deeper and conducting analysis on the social networks that learners participate in such as twitter or a blog.

Dragan (our instructor) mentioned that researchers have often thought social networks may be the most important component of learning. And the analysis of social networks is based on various research fields. He mentioned some key characteristics that will be of focus include density, centrality, and modularity.

Network Elements
Social networks have some key structural elements that can be identified in order to establish a common language and conceptual model. This allows us to analyze them.  In this mooc we discussed three key elements, the actor, relations, and data sources.  

Actor
The actor is a node or vertex within the network. In social networks this is typically a person or learner, but I don’t think it would necessarily need to be a person.  

Relations
The relation in the network refers to the ties, edges, arcs, and links that connect the actors. Relations can be undirected and weighted or they can have a direction, meaning that an actor can be the sender and any actor that receives data can be the receiver. So, actors can be senders or receivers or both.

Additionally, the relation between two actors can also be labeled or categorized.  This means they can represent something, such as friendship, advice, hindrance, or can be a form of communication. I would imagine this could be a very interesting component of network analysis to try and identify and define these relations for the purpose of understanding the learner, the network, or the context.

Data Sources
The third element discussed in the mooc was the area of how the data was collected.  I think the idea of how you gather your data will have an impact on what filter you use to analyze the data.  Is it you own data such as email or is it from twitter?  This collection process impact what you can look at and create a potential bias on how you analyze the information and what conclusions you may be able to make from that analysis.  

As a reflection, I don’t think this third element is well articulated within the mooc materials.

Analysis
There were three key areas of analysis that we are looking at which include density, centrality, and modularity.   

Density is the degree to which actors are connected to all the other members in the network. Centrality is the extent in which the actors are organized around a central point.
Modularity is the way that you quantify the modules within the network or community, by counting and analyzing the ties between the actors. (It can get more complicated quickly, but this is the core.)

Potential Benefits
My first approach to considering how studying social networks might be beneficial to enhancing learning, is impacted by traditional classroom instruction. As a facilitator you often lurk about your classroom or learning environment and just listen.  What are they saying, who is saying it, who is dominating the conversations, and do they have a grasp of concept?  Should I step in and correct something that has been miss-communicated?  Do I need to provide clues to a small group to get them headed in the right direction?  Did they understand anything I said?

I would think if you could monitor a social network of a group of “actors” all with similar learning goals (i.e. a class or group of employees) you could begin to get a sense of how their learning was/is progressing. I would also think that it meet even create a safer environment for learners who struggle with social settings.  



November 1, 2014

Getting Started with Tableau

0 comments
Collecting Data

To begin working with Tableau, I decided to pull some West Virginia Department of Education data. The WVDE has a limited set of data that it makes available here.  The dataset that I grabbed was around graduation rates. I particular, looking at the percentage of students that graduated in 4 years.

The website defines it here....

defines a high school graduate as "a student who has received a regular diploma in either four years or five years as part of the four-year adjusted cohort or the five-year adjusted cohort." Students earning high school credentials by obtaining a high school equivalency diploma or a modified diploma are not considered graduates for the purpose of the graduation data.

Integration
There wasn't really any integration with the base data set.

Analysis in Tableau
The first step is to connect Tableau to your data. I did that be simple by connecting to the excel file. I did edit the column titles to help me once it was in Tableau.

(sorry iOS users this is in Flash, I will try to find a better tool)


Unable to display content. Adobe Flash is required.

In this video I walk through a very basic set of configurations to be able to see some data.  No real analysis, but it shows you what you can do pretty quickly.

The Learning Analytics Data Cycle Breakdown

0 comments
What is the Learning analytics data cycle?   

In our mooc week 2, @gsiemens presented the data cycle shown below. It represents a cycle of what you would need to do to work with and get insight from learning data, leading to understanding that should lead to an informed action.  Therefore, a data loop.  I will break down my understanding of the data cycle below.



Data Collection and Acquisition  
This is the specific process by which the data is collected. This is very critical first step. You can start by considering what data you have, but should very quickly start to consider what data you want, but don't have. If you don't figure out how to collect it, you won't get it.  

For example, do you want what time the learning activity started?  Not when it was scheduled, but when it actually started.  How do you get that information?  Does the data have to be auto-generated or is it acceptable to have a person manually take an action to collect the data?   For me, this is more trial and error then strategic thought, but I'd like to be more proactive in the future.  Can we create simple, reliable data collection points and gather all the data this already being generated?  

I am hopeful that I can start to map some of the Internet of Things (IoT) "offerings" to this stage. Can we put a sensor in the learning environment to help collect information?

Storage
Where will you store the data. This will quickly become another "big" consideration.  When you set out on an ideal journey, you want all the data to be stored in a single location.  But you quickly realize that this is just not possible. For many organizations, the LMS was put into place with this purpose, but all systems have limitations... i.e. when did the class really start?  9:02, 9:15... when did the students login?  Does that matter to you enough to store it in another system?   

Once your in a two system world, then you are starting to into the next phase....

Integration
Data integration can get tricky, you don't want to have to keep managing the your learner demographics, but depending on your organization, even who you are measuring can be pretty fluid. Learners can enroll, not participate, enroll late, leave the learner population all together, and a million of scenarios that can drive you crazy like sharing devices, leaving in the middle of the activity...  If you have a lot of data sources, this can quickly become challenging and complex.

This is were the Data Wrangling comes in. How do you start to get data that you can structure and use?  So far, for me, this has not been a fun activity.  It takes some patience. 

Analysis
Once you have integrated your data, you can begin to conduct your analysis of the what is happening in the data. In my world, you will typically have some ideas of what you want to seek out, at least initially, however once you start to (try) to answer questions, you can move into new areas, look for new trends, look for outliers, or correlations.

It is a little interesting, most best practices seem to point to some version of the scientific method, develop a hypothesis from which to work. What questions are you trying to answer?  Do students who show up 15 minutes late perform as well as those who were present at the beginning of the class?  As an educator, you hope not.  :) Just Kidding.  Of course, there are an endless number of questions and you should really spend some time writing them down, showing them to others, and try restating them differently. I also like to tell people to think about what action they are going to take when they get the answer? Kind of a "so what" test.

I am not yet that familiar with the concepts in the map above. SNA, NLP, Concept Development. I will have to do some research to see what these things can do for the analysis process.

Representation & Visualization
This process is about portraying your analysis in a why that enhances or at least explains your analysis. Can you see trending data? Can you see relationships in the data?  Can you compare volume or velocity?  Can you look at a map and quickly know what has happened or is happening?  

This can be a very powerful moment for a group of people working together. Again, spend time looking at the story with your team, peers, or force you significant other to look at it and tell you what they think. You will be surprised at what they tell you it means.

If you can start to see these things then it is time to take action.

Action
If you are engaged in Learning Analytics to "enhance learning" (see our original definition) then you are almost obligated to take action. What are the next steps?  Based on the data that you have, what actions can be taken to enhance learning?  How does the action get implemented?  Knowing what we know now (through the analysis) can we take action and continue to measure the impact of that action?

This will take us back to "data collecting"

October 31, 2014

Defining Learning Analytics and the Insights it Can Bring

2 comments
A Definition

My task here is to define Learning Analytics. What is it? I think the easiest and broadest answer is (my definition):
Learning Analytics is the analysis of any data that was created as part of a learning process.

This definition is really wide open. It allows, or even forces, the observer to interpret almost every aspect of what that might mean and how it might be useful.  Where do you start? However, it is very non-judgmental. If you think any data relevant to learning in anyway and you want to conduct an analysis of that data, then go for it. 

SOLAR came up with this definition:
(note: it is not easily found on their website)

Learning Analytics is the measurement, collection, analysis, and reporting of data about learners and their context, for purposes of understanding and optimizing learning and the environments in which it occurs.

The first part of this definition allows you to look at four initial things you can do with data:
    Measurement
    Collection
    Analysis
    Reporting

Measurement is about the size, length, or amount of something, as established by measuring. With the data that we find in the digital world today and what is coming in the near future, we will want to leverage many measurements of data that we haven't probably thought about before in the learning context, such as those being pushed by big data volume, velocity,  variety, and veracity.

Collection is about capturing the data in a format you can use. What is generating the data? Who is generating the data? How is it stored? How "clean" is the data?  Can you perform an analysis with the data you have?

Analysis is about being able to pull insights for the data.  What does it tell us? Are their patterns? Correlations to be made?  Can items extrapolated or variables be isolated?  

Reporting is about sharing. Can you effectively communicate the insights that you found in your analysis?  Can you tell a story?  

The second part is about Learners (the person or people learning) and the context (I am assuming this is everything about the learner and/or the learning) 

The third part is about the purpose.  
for purposes of understanding and optimizing learning and the environments in which it occurs.

Maybe this will be discussed later in our mooc, but I am not completely sure why Learning Analytics would have to be for the purpose of understanding or optimizing?  I do disagree with this being the grand noble cause, but as a definition, I think it is more part of the mission of SOLAR.  I could measure learning data just know how long a module took me to do it?  Or how much it cost? 

I'm not recommending to change it, but something to think about.

Insights

This one is pretty open. What insights would measurement, collection, analysis, and reporting of data about learners and their context, for purposes of understanding and optimizing learning and the environments in which it occurs... provide the educator or the learner?

Much like having an open definition of learning analytics causes you to think broad, the insights of what we might get from learning analytics is also very broad.  I would start to build out different areas of focus, based on traditional learning analysis.  

What do you know about the learner?  Prior knowledge, Prior Experience, current level of performance, etc... However, today's connect world, we can learn so much more.  If are learners are engaged in the digital realm by using a smartphone or a tablet, engaged in social media, or using an organizational system such as a learning management system or a recognition system (think business here) then we start to gather a lot more information.   

George talks about this in our mooc video. He indicates that we can start to learning this things about the learner.
    • sentiment
    • attitudes
    • social connections
    • intentions
    • what we know
    • how we learn
    • and what we might do next


It is interesting that the "how we learn" topic continues to surface.  If we really have insight to how you learn or your learning style, can we do anything about it?  Can we create a design approach or even a smart system to individualize your learning.  

While learning for learning is awesome, my day job pushes me to have a particular interest in how learner experiences impact performance. This would help us start to have a story about how effective the learning activities were to help someone complete a task. (aka do their job).

Other Insights might include data about the environment. Classic argument of classroom vs online? When is the best time for learning? Is it better to play music when learners complete group activities? (All of these feel backward looking) Is it better to allow learners access to a smartphone throughout the learning experience?

Wrap Up

I am sure as we move forward we will be digging much deeper into these topics, but it is good to have a definition to build on. I am going to go with SOLAR for now. And to start considering all the different insights that we might get from learning analytics.

















October 29, 2014

Areas of Focus for Learning Analytics Tools

2 comments
I am participating in a MOOC around learning analytics.   We are in week 2 and I am already a little behind, but one of the week 1 competencies was to be able to identify proprietary and open source tools commonly used in learning analytics.


We were provided a tool called Learning Analytics: Tool Matrix and our activity is to add to it. The tool identifies the following five areas:

  • Data Cleansing/Integration

    • Prior to conducting data analysis and presenting it through visualizations, data must be acquired (extracted), integrated, cleansed and stored in an appropriate data structure. Given the need for both structured and unstructured data, the ideal tools will be able to access and load data to and from data sources including RRS feeds, API calls, RDMS and unstructured data stores such as Hadoop.

  • Statistical Modeling

    • There are three major statistical software vendors:  SAS, SPSS (IBM) and R.  All three of these tools are excellent for developing analytic/predictive models that are useful in developing learning analytics models.  This section focuses on R.  The open source project R has numerous packages and commercial add-ons available that position it well to grow with any LA program.  R is commonly used in many data/analytics MOOCs to help learners work with data. We opted for Tableau during week 1 & 2 due to ease of use and relatively short learning curve.

  • Network Analysis

    • Network Analysis focuses on the relationship between entities.  Whether the entities are students, researchers, learning objects or ideas, network analysis attempts to understand how the entities are connected rather than understand the attributes of the entities.  Measures include density, centrality, connectivity, betweenness and degrees. 

  • Linked Data


    • If Tim Berners-Lee vision of linked data (http://www.ted.com/talks/tim_berners_lee_on_the_next_web.html) is successful in transforming the internet into a huge database, the value of delivering content via courses and programs will diminish and universities will need to find new ways of adding value to learning.  Developing tools that can facilitate access to relevant content using linked data could be one way that universities remain relevant in the higher learning sector.

  • Visualization

    • The presentation of the data after it has been extracted, cleansed and analyzed is critical to successfully engage students in learning and acting on the information that is presented.  

My next focus will be to identify key tools within each area.


October 22, 2014

Are you aligning Training with Performance?

0 comments

Focus Training on Performance

As a learning leader it is important to understand how to best align your training activities with the performance goals of your organization.  To do this, we need to be sure that we always keep performance as the goal, communicate transparently and clearly, and think of learning as a process, not an event. This can be challenging, because a lot of we do looks and feels like an event.  

It feels like those of us who analyze roles, design and develop curriculum, facilitate a course, and evaluate the success for that course, assume that we have "performance" as our main focus.  But unfortunately, it can be elusive.  We can find a great concept and really focus on delivering that concept, hoping the learner will take it back to the workflow they live in on a daily basis.

Measure the Learners Performance

We also have to be careful that we don't get caught up in worrying about measuring ourselves.  When we look at the data we have, we tend to want to focus on how training activities went, how did we do, did they like us. But this takes the focus away from the performance of the employees. Can they perform the task(s) we taught them?

Communicate their Performance to the Field

We need to be able to communicate clearly to the business unit...

here is how your staff is performing and if at all possible, be part of the performance communication when they are back in the field.

Are you aligning your training with performance?





September 13, 2014

Learning about Big Data

0 comments
The emerging field of big data is being hyped everywhere. I too, am pretty excited about what is happening and even more excited about the potential it represents. The big data movement is primarily about leveraging unusually large amounts of data in a way to make better decisions.

Some of the more notable examples being referred to include Moneyball, where the data showed that on-base percentage was a better predictor of runs, then the traditional batting average in Major League Baseball and the NetFlix competition for a movie recommendation system and similarly Amazon’s website.  Others that may not be as notable, such as IBMs work with a beverage retailer that looked at hyper local events (social media postings about soccer practice ending), the weather (sales above 70 degrees), and other relative data points. In this case they analyzed over a billion points of data that could potentially impact consumer behavior.

Today, there is more and more investment in capturing and analyzing data. Communities like these are getting more popular.  These are interesting times.