LDT506 Unit 1 Self-Assessment and Reflection

 U1: Assignment - WOL: Self-Assessment and Reflection

For Part 2 of this assignment, you'll write an 800- to 1,000-word Reflection that you'll post to your WOL Blog. If you haven't created a WOL Blog in LDT 501 or LDT 502, please refer to the directions in the Working Out Loud module in this course to create one. Since blogs don't provide a good mechanism for determining page length, it's strongly recommended that you use Microsoft Word or Google Docs to draft your assignment.

Self-Assessment and Reflection

For LDT506 Evaluations of Learning System(s).  A course for my Masters in Education.  We have been given the assignment to do a self-assessment and reflection.  Along side this assignment, we have been given this really cool tool JASP.  Given that the two assignments where in the same week, I figured it would be a good exercise to combine the two assignments and get a little more experience with both.  With JASP, I had to play around with a couple different data sets to try to get a feel for how this program works.  Being a statistical geek, I was curious what I can do with this application. 

Because I only had a single student to work with, I had to format the data several different times to see how it would give me results.  I could not figure out how to get JASP to work how I would normally format the data, because I only had a single student to work with.  Normally I would put each student as a single row.  But I could only figure out how to compare one row to another row which doesn't work for a single student.  So I ended up putting each student into a column.  In this case only a single column and then added a column for student name number domain and domain name.  So data set looked like the following:

For the Whole test.  I had the following datasets

Valid Questions 49
Missing 0
Mean 3.061
Std Deviation 1.725
Minimum 1.0
Maximum 6.0

For each Domain I had the following number of questions

More base questions on Methodology and less on Interpersonal and Context, so that could sway the results based upon how the Methodology section gets answered compared to Interpersonal and Context.

Lets see how the frequencies play out?  How did I answer the whole evaluation versus the individual domains?  First the total self assessment and then I'll break it down by domain and see what my strengths and weaknesses are for each domain.

Give 15 counts with an answer of 1 (A novice), and 6 counts of 6 (An Expert).  Lot more of being a novice than an expert.

So Really I had 6 questions where I felt myself an expert or qualified to be an evaluator, and almost 3 times as many where I felt myself a novice.

Breaking it up for the different sections of Professional Practice, Methodology, Context, Planning, Management and Interpersonal.  Which should help me with the answers of this  paper, how to identify some of the skills I might already have as an evaluator.

So for the different Domains:

My mean ran from 4.11 in professional Practice to 2.375 in Interpersonal. So clearly I'm higher in Professional Practice but much lower in Interpersonal, which is probably my own observation.  From a standpoint of how I view myself from a professional standpoint and how I deal with things in my own life I am much more comfortable applying evaluations to, and putting in them in practice for my job, than I am with Interpersonal situations.  However I'm not sure that's a horrible thing given my profession.  I currently don't interact significantly with a student population.  However I do work with multiple and sometimes large data sets on a regular basis.

So with the above data set using the sample from EDT 506: Course Entry Self-Assessment - SP23.  Let me try to figure out where I answered 6 in the different Domains.

Professional Practice

In Professional Practice the questions tend to surround how I can relate my job to that of an evaluator and what are the different professional situations that an evaluator should be able to deal with. 

Breaking down my answers in the professional practice section of the Self-Assessment. There where no middle of the road answers.  Either I felt like I knew what I was doing or I didn't.  Clearly either I didn't understand the context of the question, or felt like I was a novice when it came to being able to answer that question.

Of the questions I felt like an expert:
Question: 1.2:  I am able to apply the foundational documents adopted by the American Evaluation Association that ground evaluation Practice.  So clearly taking a specification and applying it I'm comfortable with.  I'm not completely certain of the the foundational documents.  I hope to get a good understanding of them in this class, but I feel confident that once I understand these standards I can apply them.

Question 1.4: I am able to use systematic evidence to make evaluative judgements.  The way I read that question:  Can I interrupt results from a data set?  Yeah, I'm comfortable taking a data set and making an evaluation.  Perhaps it's incorrect, but that's part of the learning process.

Question 1.5: I am able to reflect on evaluation formally or informally to improve practice.  The way I read that question was:  Can I do an analysis of the evaluation and change it around based upon what you are trying to ascertain.  I feel comfortable with that. 

Question 1.7:  I pursue ongoing professional development to deepen reflective practice, stay current and build connections:  Am I constantly trying to learn?  Yeah, I'm always trying to learn new things and new ways to develop professionally.

Where I need help with identifying personal areas of professional competency include  promoting social justice and public good,  and determining the field of evaluation and its value.  I guess this course will be a good one for me to take.  An initial observation on the course as a whole is how social justice and public good are even mentioned in evaluations.  I'm currently not really sure why those concepts would ever be applied to evaluations.  From my perspective, I'm not going to change an evaluation based upon social justice or public good.  An evaluation is an evaluation.  If a particular group doesn't like the results, either I didn't have a complete data set including that group, which should be noted in the results, or they can go pound sand.  I'm not changing data to appease a particular group of people.  Perhaps my opinion is too rigid, but numbers don't lie.

Methodology:

Methodology is the technical aspects of an evaluation.  How to measure something and how to break down the answers either quantitative or qualitative and how to interrupt those results.

Breaking down my answers in the Methodology section of the Self-Assessment. Almost all answers where middle of the road.  Either I didn't feel like I knew what I was doing with confidence, or didn't feel like a novice for very many questions in that domain.  Again perhaps I didn't understand the context of the questions. but for the most part I was fairly confident but not at the expert level.  My own personal opinion is that I see many different ways to grow in my methodology.

Well only a single 6 on Methodology.  

Question 2.11:  I am able to analyze data using credible feasible and culturally appropriate procedures.  There I think it was asking if I have the culturally appropriate procedures clearly defined which gives me the ability to say I can analyze the data.  If I was going to have to define the culturally appropriate procedures, I would probably not have answered 6.

I'm a little concerned with only one count of 6 in the Methodology and think I might not have been reading the questions in the contexts of the Domain.  I'll probably go back and take the test again with a different mindset.  However given 3 days to analyze the data.  I'll go with this initial sample.  I think this sample set really goes to the core of question 2.12.  I am able to identify the strengths and limitations of the evaluation design and methods. 

What it appears I need work on, or felt like a novice for Methodology are evaluation purposes and needs, being able to determine evaluation questions, and being able to design credible and feasible evaluations for a specific reason.  I see this last one as being really valuable.  Being able to design and create a evaluation that addresses a specific purpose is something I really should work on.

Context

The context domain is how to deal with the evaluation within constraints.  Plans, timelines, resources as well as environment and subjects.  The Environment is one I'm going to struggle with.  Some of the sub categories of environment include Culture and Diversity, as well as Values, Beliefs and privilege.  In my opinion many of these categories have no reason to be included in a Evaluation.   But, I have an open mind and hopefully in this course, I'll come to better understand why some of those topic would be included in an evaluation.  Perhaps it's meant to compare privilege levels?  Or to contrast one data set between two races or cultures?  In that particular instance I would agree it would be valuable information.

Breaking down my answers in the context section of the Self-Assessment. There where no expert answers and only a single novice.  So mostly middle of the road. Slightly below average.  With an average of between 3 to 4 , halfway between novice and expert.

For the Context domain I didn't have any questions where I felt like I was an expert and only a single question where a felt like I was a novice.

Question 3.8:  I promote evaluation use and influence in context.  My thoughts on this:  Why would I allow influence on an evaluation?  Can we allow evaluations to be influenced based upon context?  That seems like manipulation of the data to create a desired outcome.

Planning and Management:

Planning and management of an evaluation.  This is using the constraints and components from the methodology to create an evaluation study.  Basically what a Project Manager in my line of work does.  Takes all the available resources and they figure out when and how to apply the resources (Labor, equipment, locations) to complete the project.  I'm not really a project manager.  I understand what they do, and have been involved with hundreds of project plans but strictly as the technical resource, defining the resources, not detailing how they are all put together, and that tends to show.

My planning and management foo is weak.  When it comes to project planning I felt like I was a novice on most of the questions.  Stating in general I'm not a project manager.  I understand resources, but planning the use of those resources, I need some work.

I did feel like an expert relative to only a single question in the Planning and Management domain.

Question 4.10:  I am able to use technology appropriately to support and manage evaluations.  Yeah, I understand technology, and generally can learn how to apply that technology at a fairly quick pace.

Interpersonal:

The self Assessment defines Interpersonal as the human relation and social interactions through cultural competence, communication, facilitation and conflict resolution.  To me this has always been the touchy feely parts of the Human Resources department.  Stuff I generally try to avoid and it kind of shows.

Analysis of Interpersonal.  Mostly novice and lower middle of the road.  I had 3 questions where I felt like a complete novice

5.1 I am able to foster positive relationships for professional practice and evaluation use.  Creating relationships for evaluation.  I'm not sure I completely understood this question, but I read it as using relationships in the evaluation.  Something I'm not entirely sure how this would happen.  I related it to my work as getting somebody to take a course or complete some action.  Which if I'm not their direct boss, I do have a problem dealing with that situation.

5.2 I am able to attend the ways power and privilege affect evaluation practice.  I'm still just not sure how this impacts evaluations.  The way I read this was that because I'm a middle class white male age 55, I'm not qualified to create evaluations for the poor black population.  In which case I completely disagree.  But, then perhaps that's a much more complex conversation.  However if the case is in this question:  Can I understand how a particular question might be swayed based upon power or privilege then yes, wording a question a particular way could cause problems with the results.  But again, I'm not an expert on how to change the question to a manner in which it's appropriately worded, again I'm a novice.

5.7 I am able to facilitate constructive and culturally responsive interaction through the evaluation. I'm not sure if I answered the question, or what exactly the question was asking.  An evaluation shouldn't be swayed based upon culture.  It should however point out the differences that culture makes on the evaluation.  But that's an opinion, I'm trying to learn or have a more informed opinion.

Interpersonal  in general is probably my weakest part of the self-assessment.

Final thoughts on my Self Assessment

So after having taken the Self-Assessment and reflection  I ranged in the domains from 2.375 to 4.111 with my average being 3.061 Slightly below median of 1-6.  I felt more comfortable but less of an expert on the Methodology and mostly like a novice on the Interpersonal domains.

Real interesting project, gives me a lot to think about, and stuff to contemplate while reading the textbook Evaluation in Organizations.

References:

Evaluation in Organizations Second Edition (2009), Darlene Russ-EFT, Hallie Preskill

AEA Guiding Principles https://drive.google.com/open?id=1SH3toIADrR6hyMQyvzAiHMAAqdbZ8aC-

AEA Evaluator Competencies https://drive.google.com/open?id=1B7GZrhi67Fbt_nOW7jizJ-RnlJ8w-P3J

IBSTPI Evaluator Competencies  https://drive.google.com/file/d/1UPG1CPr1yIQzqCbRuacEpZaejmCBjtNe/view