Vardaan Shah

Design Revision 1

Introduction

After making the demo usable, I conducted interviews with four users to get their thoughts on some particular design questions I had (included in the main background section). In addition to these design questions, I asked the interviewees about their thoughts on the software as a whole and whether they had any specific feedback to give independent of the questions I asked. From this feedback, I've extracted a set of design parameters and made a corresponding set of design decisions. What follows is an accounting of the decisions I made and how I came to those decisions.

Scoring Function Adjustments

The scoring function is at the heart of the MedPet concept. The pet itself is a way to present the scoring function to the user in a way that makes it engaging. As such, the function's design is of utmost importance. In order to understand the design space of the scoring function, I've provided here a description of the scoring function. It is not necessary to understand the entire function to understand the design space, but if further context is desired, this description will provide it.

Scoring Function Description

The scoring function only tracks the status of one medication being taken, i.e. if a user needed to track four medications, the MedPet would internally maintain four instances of the scoring function. It would then proportionally combine the score from those instances into a single meta-score. Since the meta-score is a linear combination of several instances of the scoring function, the scoring function and the meta-score need to have the same desirable properties.

The scoring function, mathematically, needs to have its output be restricted to be between constant maximum and minimum values. This allows the design space of the rest of the MedPet to be constrained.

The scoring function takes, as input, two sets of data:

  1. A list of times that a particular medication was expected to be taken (E)
  2. A list of times that that medication was actually taken (A)

Internally, the scoring function has a comparison function. This comparison function takes 2 inputs:

  1. A single time that a particular medication was expected to be taken
  2. A single time that a particular medication was actually taken

It outputs a number between 0 and 1 representing how close the actual time was to the expected time.

To actually output a score, the scoring function generates a table with E rows and A columns. Each cell in the table is populated by the result of applying the comparison function to the corresponding actual and expected time the medication was taken.

For each row, the highest score is selected.

A windowing function is applied. The windowing function takes 2 inputs:

  1. The score selected from a row of scores
  2. The difference between the current time and the expected time that corresponds to the score.

The function applies some function to the time difference to get a "relevance multiple" (RM). RM represents how much the past should impact the score.

The windowing function then outputs the selected score multiplied by RM. This windowed score is added to a running sum.

After the windowed scores from all the rows are added to the running sum, the scoring function accounts for anomalies.

There may be multiple actual times taken that correspond to the same expected time taken. A windowed penalty is applied to the running sum for each extra actual time assigned to an expected time. There may be expected times that have no corresponding actual time. A windowed penalty is applied for each of these.

The scoring function then outputs this adjusted sum as the score.

Design Space

There are several generic functions in the scoring function that are spaces for design.

The comparison function represents how close a user was to taking the medication at the ideal time. If there is no difference between the time the medication was supposed to be taken and when it was taken, the comparison function will output 1. I can adjust how quickly the output reaches 0, and in what manner.

The windowing function represents how far into and to what extent the user's past performance should have an impact on the score. The faster the scoring function decays to 0, the less the scoring function will take the past performance into account. The longer the scoring function takes to decay to 0, the less responsive to current change the function will be.

The penalties for excess medication taken or medication not taken are also a parameter that can be adjusted.

Final Decisions

The windowing function will decay exponentially from the present time through approximately 1 week in the past.

The comparison function will be approximately parabolic in shape.

The comparison function will remain at its highest value for difference values from 0 to at least an hour in either direction.

The penalty for excess medication taken will be at least 1.5 times the penalty for missed medication.

Rationale

Currently, the windowing function is set to be 1 when the time is within two weeks, and 0 otherwise. All four users struggled with this. In particular, users struggled with the fact that low or negative scores would often linger for longer than seemed reasonable. All four users attempted to find alternate explanations of their score remaining low. When those low or negative scores finally stopped mattering, the score jumped back up very suddenly. As a result, feedback on the user's actions were always about two in-game weeks delayed. This caused significant confusion and prevented the users from grasping the mechanics of the scoring function. An exponential decay in the windowing function will make it so that it is clear to the user that their present actions are more important than their past actions. In addition, an exponential decay function would prevent the problem of the score suddenly jumping up or down for no discernible reason when the time that the meds were supposed to be taken fell out of the window.

The parabolic shape alongside its clipping to the highest value for 1 hour in either direction arose from feedback from User 3. User 3 in particular was extremely particular about maximizing the score that they could receive from taking their medication on time. They were the only user who used the "jump forwards in time" functionality in the demo. They paused time at almost exactly the start of an in-game day, and then jumped forwards in time by exactly one day several times to receive the highest possible number of points from each instance of taking the medication. I can see users who have a similar instinct towards optimization or obsession struggling with a scoring system that has no margin built-in. In addition, anxiety disorders have a prevalance of approximately 25-50% in patients with ADHD [^1]. It seems important to build in that margin, since it is likely that people prone to rumination would be a significant portion of the potential user pool.

[^1]: Koyuncu, A., Ayan, T., Ince Guliyev, E., Erbilgin, S., & Deveci, E. (2022). ADHD and Anxiety Disorder Comorbidity in Children and Adults: Diagnostic and Therapeutic Challenges. Current Psychiatry Reports, 24(2), 129–140. https://doi.org/10.1007/s11920-022-01324-5

The choice to penalize excess medication more than missed medication came from feedback from User 2. User 2, at some point in the interview, had missed a few days in a row. As an attempt to recover their score, they took several medications at the same time. Because the penalties were not balanced appropriately, this actually resulted in an increase in the score. Most medication is not designed to be taken in excess of the therapeutic dose. Stimulants especially, as they have a higher potential for addiction than many other medication, should not be taken in excess of the prescribed dose. It is, therefore, extremely important that the MedPet does not communicate an incentive to take excess medication, even when a dose is missed.

Scoring Function Permissions

Many medications have different tolerances as to when they should be taken for the best results. In addition, I think that different people respond in different ways to score-based incentives. Thus, I felt that it would be helpful for the scoring function to be adjustable. However, this presents a question: who should have the power to adjust the parameters of the scoring function?

Final Decisions

Patients should have the ability to adjust the comparison function

The manufacturer should offer 2-3 discrete options for the decay rate of the window function.

When the scoring function is altered, the medications intended to be taken prior to the change of the scoring function should still be scored under the old function.

Rationale

All four users expressed a relatively weak preference for who could adjust the scoring function. Some themes emerged, however. Firstly, User 1, User 2, and User 3 all mentioned that time with a medical professional is scarce, and expressed concern that if adjustment were limited to a medical professional, it would limit the MedPet's usefulness.

User 4 had this to say:

I'm thinking of like other utilities that currently exist for people to just simply take medication, and I guess how those systems currently work is the doctor, or like the person prescribing the medicine, simply just tells the patient: "hey you should take it at these intervals." Then, the onus is on the patient to actually enforce that for themselves. I guess here, MedPet is taking the place of that, helping the patient take accountability. I think in that sense maybe it should be whatever the patient finds useful to themselves.

User 4 brought up an excellent consideration here that I did not consider adequately: medical providers, especially for outpatient care, are not held responsible for their patients' adherence to a treatment program. It makes little sense, then, to limit the adjustment of a tool designed specifically to help patients adhere to a treatment program to those same medical providers.

User 2 said, in reference to a user being able to change the scoring function:

If you could go in and easily change the difficulty then it could lend itself... I could see myself falling behind on it and then being like, "um, I'm going to easy mode now."

Fundamentally, this is the tension I am grappling with. The user should be unable to change the way the pet works because they feel negatively about their progress, since this would provide a way to resolve that feeling without actually becoming more consistent. At the same time, the user should be able to adjust the parameters of the scoring function to align with the actual characteristics of the medication they are tracking, i.e. a medication that they need to take exactly on time should score differently from a medication that can be taken in a 6-hour window with the same effectiveness.

The decay rate of the window function is one of the ways in which the scoring function's global "difficulty" can be adjusted. MedPet's goal is to help the user align their behavior with their intentions using the MedPet as a measurement/feedback device which is calibrated to the intention. Unrestricted user-initiated difficulty adjustment threatens to instead calibrate the MedPet to the present, problematic behavior.

One of the ways to discourage this reverse calibration is to essentially set the past as immutable. If changing the scoring function only changes how future events will be scored, it substantially reduces the risk of a user changing the function just to feel better about their immediate choices (it won't work), and instead makes it more likely that the user will adjust the function to what they believe they can achieve.

It is important that the user be able to adjust the comparison function because each medication is truly different, and incentivizing "correct" usage looks different from medication to medication.

Discrete options reduce the ability and desire to endlessly fiddle with a setting to get it perfect instead of using the product as intended. Providing discrete decay rate options allow users to choose the curve shape best for them without presenting them with another parameter to tinker with.

Personalization

A significant portion of the emotional bond that Tamagotchis induced in their users was the users' ability to customize them. However, another important aspect of that bond was experiencing the illusion that the Tamagotchi was "alive" in some way. As such, there is a balance to be struck between building up the MedPet as a distinct, identifiable object, and allowing the user to feel like they had some part in the MedPet's creation.

Final Decisions

The user should be able to name the MedPet upon starting its use.

The user should be unable to later rename the MedPet.

The user should be able to freely customize the visual appearance of the MedPet with accessories or similar.

The user should not be able to adjust the core "personality" of the MedPet.

Rationale

A theme that emerged through discussions with all four users is the distinction between "essential traits" and "auxiliary traits." All four users agreed that, however they framed those essential traits, essential traits should not be mutable. They also agreed that the auxiliary traits should be freely mutable.

User 4 said (emphasis mine),

I think it's probably fine to make it so that they can't change it [the pet's name]. If you really want to add a feature where like you can change the name, maybe make it to the point that like you're not changing their name, but you're like having to like give them away and like get another pet with like default stats or something like that...Like, making it actually feel different.

User 2 said, in response to me asking whether allowing arbitrary name changes were a good idea:

I think that would be detrimental because...I feel like I would have a harder time getting attached to it.

In particular, what interested me was the idea of making the MedPet feel different if renaming it was desired. It seems that, to Users 2 and 4, the name of the MedPet would be an essential feature. User 3 also agreed that the name should not be alterable.

In contrast, all four users agreed that customization of the visuals of the MedPet was important to feel attached to it. User 2 brought up an interesting point in this regard:

Even with, like, a real pet, you could go take it and give it a haircut. So that's less important.

User 1 added this:

Like, specifically, like, as you interact with the pet more, more customization options open up. Because, obviously, people love to customize things and make them their own.

Although I don't think I can make a concrete design choice from this insight, I think what User 1 said here is important: the MedPet should have some kind of long-term interaction potential.