Integration keen.io with trello script

So I spent some time yesterday to integrate the trello script with keen.io

  1. Data collection
    keen.io for those who are not aware is a platform which provides APIs to collect, analyze and visualize data.It typically stores data in something called collections. So I made a collection called performance to which the script(which would usually run daily as a cron on a machine) would post data. The data contains the name of the team member and their score calculated that day looking back a week. This data would be posted conditional to the presence of a flag while running the script. Also, if the data is successfully posted to keen.io, a notification would be sent to Slack. This way we would know the script ran for the day. If we don’t get that notification, we can check the machine on which the script is setup to run and see if something went wrong. This part is done.

  2. Data Visualization
    For this keen.io provides a Javascript SDK, which can read data from a keen.io collection and plot it in a graph. I think I have figured out the right query for this, but just need to test it out for a couple of days. An issue here is that keen.io doesn’t host graphs/dashboards. We have to host them ourselves. Github pages could be a free and easy option, but then that would be publicly available because that’s how Github pages work. So I have to find out how to host this with the minimum overhead of maintenance.

  3. Setting up cron
    After the first two steps are done, I will setup a cron that runs at the same time every day.
    Qn - Do we want it to run on weekends too ? If it doesn’t then the graph would be discontinuous. Other than this, it should be easy to set it up.

Yeah, that’s the biggest issue with keen.io. You can look at the third party integrations:
https://keen.io/docs/data-visualization/

Otherwise, we could just publically host it as part of dgraph.io/perf. Would be radically open and transparent! :slight_smile:

1 Like

I wouldn’t mind us publically hosting it :slight_smile:

I’m okay too!

@ashwin95r, @koppula: What do you guys think?

Btw, @pawan: Can your Cron also post the summary on Slack, instead of just pinging it?

Actually the trello script pings slack.

Sure, but do we want the summary everyday?

Yup. Summary everyday, just like what gets posted to keen.io

Is it just the score that is public? or are the name of the tasks that were performed publicly visible too?

It will be a graph with scores overtime that would be public. Not the tasks

@mrjn Ok, I was thinking seeing a daily score won’t add much value(a weekly score could through) whereas a graph of scores over time would. Though sending it to slack is pretty trivial.

I meant a weekly window published daily. That way you know if you are regressing, and can do something about it during the week.

Thanks @mrjn. I understood that earlier too. That’s the same score that’s sent to keen.io.

1 Like

Ah… okay. I wasn’t sure. Though, it’s better to over-communicate and be absolutely clear :smile:.

Btw, you might want to push out your changes so they don’t just stay on your laptop. I push out to git multiple times a day to avoid any hard disk crash disasters.

1 Like

Sure , I will do that today. I should make that a habit, a hard disk crash would indeed be painful.
I may not have access to the tools repository though.

DGraph team has write access. So, you have the access to commit. Though eventually you might want to send out a pull request, so I can have a look as well.

Yes, sure I will send across a PR as soon as I get to work.

1 Like

Checked out datahero. They are good but damn expensive. I have just picked one of the open source dashboard themes from the keen.io page. If everyone is ok, we will host it on Github pages and be good to go for now.

If we’re going to be public about this, no point paying at all. We can host it on Github, or we can even just run it as part of our website – a separate page in there or something.

@koppula @ashwin95r Your thoughts on having this dashboard public ?

I think hosting on Github makes sense. Not so sure about putting this up on the website though – I think we don’t on the website and just on Github.

While it can be public, I don’t think we need to put it up on the website or link to it from other places.

Cool, will host it on Github pages for now.

We are still refining the formula we use to calculate this score and are still to come to up with a threshold of what constitutes a good score. So I also agree we don’t need to advertise it. Maybe when we publish our blog post about our trello integration, we could add a link to the dashboard then.