API overview (pre-release)
Hello Developers!
I want to get the ball rolling on the upcoming API. We've been fielding a lot of private inquiries and I'd like to give a high-level overview of where things are at and where we're headed. A proper knowledge base/faq will be up once things reach a more-or-less steady state.
The API is REST-y.
- All requests are via https, GET for reads, POST for writes (create/update/delete).
- All data is returned in JSON format, utf-8 encoding
Syntax
- GET /?api=v1&token=your-auth-token&method=method-name¶m1=value¶m2=value
- POST /?api=v1&token=your-auth-token&method=method-name [ form data as x-www-form-urlencoded key/value pairs, or json=obj ]
GET Methods (still subject to change)
- auth
- user
- prefs
- tags
- tasks
POST Methods (still subject to change)
- saveuser
- savepref
- savetask
- savetag
- emptytrash
Example Response
{
"build": 999,
"stat": "ok",
"requesttime": 1281021590,
"results": {
"tasks": [{
"id": "12345",
"parent": "0",
"type": "0",
"state": "1",
"suspended": "0",
"completed": "0",
"cancelled": "0",
"seq": "2",
"seqt": "0",
"seqp": "3",
"name": "Call Mary re: plans this weekend",
"tags": "personal,phone,mary",
"etime": "5",
"energy": "1",
"waitingfor": "",
"startdate": "",
"repeat": "0",
"repeatuntil": "",
"repeatfrom": "0",
"duedate": "20100805",
"note": "home phone: (212) 555-1212",
"created": "1280867871",
"updated": "1280948911",
"deleted": "0"
}]
}
}Javascript Same Origin Policy / Cross-Origin Resource Sharing (CORS)
- An optional ?callback=function_name parameter will return the data wrapped as JSONP.
- Response headers are set to Access-Control-Allow-Origin: * for browsers that honor it (e.g. Chrome, Safari, Firefox, IE8)
I've attached a simplified diagram of how we're separating what was one giant tarball of a project into distinct sub-systems.
OK, questions, comments, suggestions, complaints? Let's hear em!
-
logical.png
56.1 KB
logical.png
2 Posted by haeffb on 05 Aug, 2010 04:39 PM
Elbert,
Looks great to me. A few questions:
1) will saveuser be able to create an account if not exist?
2) do you plan to add a "custom" field to tasks as previously discussed?
3) use savetask (etc) to both create a new task (id="" ?) and edit existing?
4) what are potential values for repeat? will it handle "third sunday of November every year"
5) will you use, for example, an md5 hash of password in authorization so that I don't have to save it on device as plaintext? - see Toodledo API for example.
6) will tokens expire? after how long?
7) limit on API requests per hour, day, etc?
x) is it live today so that I can begin coding/testing!?!?
Support Staff 3 Posted by Elbert McLaughlin on 05 Aug, 2010 04:50 PM
@haeffb - I was hoping you'd chime in. :-)
Answers:
1) not initially (ever?)
2) thanks for reminding me, will do
3) yes. if an id is 0 or blank, a new task is created. if the id is set, sent values are updated, omitted values are untouched. if the id doesn't belong to you, the api will bark.
4) suggestions please... we're still thrashing this out and hate every "solution" we've come up with thus far (which is why it's not implemented on the live site already)
5) yes
6) yes. as for duration, what do you suggest?
7) wouldn't it be nice to not have to? we'll have to figure out a low-impact way to address potential abuse unfortunately. implementation suggestions?
x) maaaaaaybe. ping me privately.
Support Staff 4 Posted by Elbert McLaughlin on 05 Aug, 2010 04:55 PM
Rethinking answer 1) .... perhaps we should allow this. Would need to reconsider account creation based on username, rather than email address (which could be verified after account creation, with the caveat that Nirvana would prohibit email notifications until the address is verified).
5 Posted by haeffb on 05 Aug, 2010 05:25 PM
The ability to create an account from device is a handy way to drive more users to Nirvana. In my Toodledo app, I require entry of email / password on first launch - if they don't have an account the Toodledo API will create one. Avoids users getting upset that they have to go elsewhere to use the app. Of course, they're more upset with me than with you...but guilt by association and all that. I'd suggest you give it more consideration - especially if/when you develop your own iPhone app.
I'll have to give some thought to how best to handle recurring tasks. I presume you've looked at Toodledo, which is the only other tasks API I've used extensively. They have "normal" repeats and "advanced" repeats, with the option to let Toodledo handle advanced repeats rather than on device. So, I can take care of daily, weekly, etc recurring tasks and let them handle "every Sunday". It works well for me. But I'm positive that your users will want to be able to do advanced repeats.
Duration for auth token? 4 hours seems like a good compromise. I have no clue of the security implications, if longer is worse...?
Sorry if it seems like I'm going on about toodledo, but that's where my experience lies.
I don't think I have a direct line to you - email me or you can often find me in #webos on irc.freenode.net.
Finally, how are projects and areas handled?
6 Posted by haeffb on 05 Aug, 2010 05:26 PM
And smart lists? would be nice to be able to sync those as well. When/if they're fully implemented.
7 Posted by Alex on 06 Aug, 2010 11:37 AM
What about Areas of Focus? Where does that go in the API?
-Alex
Support Staff 8 Posted by Elbert McLaughlin on 06 Aug, 2010 02:09 PM
@Alex -
We are trying to make the API really easy to work with.
Areas (and Contacts) will both become a tag variant... or so is the plan.
Entity: Tag
We want apps to have the flexibility of filtering on tasks from any entry point, be it Areas (home, work) or Contact (john,mary) or Free-standing tags (email,phone).
Also, we're trying to "flatten" entity relationships as much as possible, to make building offline apps that sync-up later as painless as possible. To that end, "tags" on a task will be a comma-delimited list of short tags.
task.name = "Some Action To Do"
task.tags = "home,phone,john"
This way, if a new task is created offline (no task.id) and new tags appear on that task (no tag.id), task to tag relationship won't break. The "foreign key" if you will is tag.short.
Make sense? Bad idea?
9 Posted by Alex on 10 Aug, 2010 02:38 PM
Hi Elbert,
I don't know enough about this subject to be particularly insightful (I am not designing an API, i.e.) but thanks for the excellent detail on this topic. I'm sure it will be helpful to somebody.
I basically wanted to make sure that the excellent Areas of Responsibility (AoR) is not forgotten in the API, and I am reassured to see your explanation (i.e. type - basic | area | contact). I think that sounds good.
This is a great platform and I hope it gets wide adoption!
Keep up the excellent work. -Alex
10 Posted by haeffb on 30 Aug, 2010 07:02 PM
Just wondering what's up... :)
Support Staff 11 Posted by Elbert McLaughlin on 30 Aug, 2010 08:12 PM
yes yes... we've gotten bogged down with other things, html5 offline being key among them. kinda threw a wrinkle into our previously ready-to-go API. really really hoping for a mid/late-september rollout.
Support Staff 12 Posted by Elbert McLaughlin on 30 Aug, 2010 08:17 PM
i might as well be more specific, seeing as this is the API thread after all. here is what has been changed:
everything is UUID based... so that new entities (tasks for instance) can be generated offline without a server id lookup
every field on a task now has a corresponding timestamp to denote when it was last modified... so that tasks can be synced without conflict (example: task marked complete on the iphone while offline, meanwhile notes were appended via desktop...)
you get the idea. that's the biggest change.
13 Posted by Jakob Heuser on 30 Aug, 2010 10:55 PM
1) Will the system be built on top of OAuth2?
2) In addition to JSONP and CORS, will you allow Flash based XHR for older (ie) browsers that don't have CORS support yet?
3) Will users receive a UUID like tasks/projects (to help track delegation and teamwork in external implementations)
4) Given every field has a corresponding timestamp for its last-modified, will we receive an error code when we attempt to post to a conflicted field without having resolved it?
Awesome to see the API coming along. I'm looking forward to playing with it once it's publicly play-with-able.
14 Posted by parakoos on 04 Oct, 2010 10:16 PM
I'm guessing the mid-to-late-September estimate is falling a bit by the wayside. ;-)
I'm keen to make an Android application to back up Nirvana, but I don't want to start until I know what the API will look like, or before I know that it will for sure happen.
Any latest news on the development of the API?
Parakoos
15 Posted by Siraj on 27 Oct, 2010 06:03 AM
Any update on the API?
16 Posted by beau on 15 Nov, 2010 09:24 PM
I'm very interested too--would love to be able to play around with this; I've got so many ideas already. :)
Support Staff 17 Posted by Elbert McLaughlin on 15 Nov, 2010 09:30 PM
well... the update is that the API is now being put to the test with Nirvana 2. not documented yet though. give us a week or so see how the dust settles from this sneak preview experiment? sorry to keep draggin this out...
18 Posted by jakob+nirvanahelp on 15 Nov, 2010 09:49 PM
Thanks for the update Elbert. Any leads on the questions to #13 above? Especially if a crossdomain.xml file will be available (for things like Adobe Air applications).
Support Staff 19 Posted by Elbert McLaughlin on 15 Nov, 2010 10:14 PM
@jakob - hadn't thought of that, but by all means i'm sure we can figure something out.
20 Posted by beau on 16 Nov, 2010 08:12 PM
Hey Elbert--I couldn't help myself and played around a bit with the API (no OAuth, of course, but I used the same method that the web client uses to obtain an auth token).
Should have a nice visualization (my goal was to make graphs that show how long tasks stay open before completion or something like that) to show everyone in the next couple of days. :)
Support Staff 21 Posted by Elbert McLaughlin on 16 Nov, 2010 08:35 PM
@beau - ha! youdaman!
can't wait to see what you came up with.
22 Posted by Kenny Grant on 07 Dec, 2010 04:31 PM
@beau
Me too.....how are you getting along?
23 Posted by beau on 08 Dec, 2010 07:11 PM
@kenny: Going well, haven't had much time to put into it lately though!
How about you?
I had another idea for an API project, though--a kind of "Venn diagram"-esque view that showed your projects divvied up inside different tags or in different Areas of Responsibility
24 Posted by Kenny Grant on 09 Dec, 2010 08:15 AM
@beau
Same here really - not got anything done, waiting for Nirvana 2 API to become available and then explore some ideas.
Venn diagram sounds interesting - look forward to any further news.
25 Posted by Shiki on 26 Jan, 2011 02:09 AM
Hello guys, is the API open? :) Can we play with it?
Support Staff 26 Posted by Elbert McLaughlin on 27 Jan, 2011 07:44 AM
@Shiki - I can tell you that I'm personally very frustrated that we have not been able to get the API into the wild, despite the fact that it's fully functional and proven, in that Nirvana 2 runs 100% off the API without problems.
Part of what we've been bogged down with is getting Nirvana migrated to better cloud hosting. We need more capacity (in particular requests per second) and better request throttling... to ensure that any 3rd party apps running against our API won't be able to inadvertently overload our systems. Think Twitter's frequent fail whale page... kinda cute for a non-critical service like Twitter, but I'm not so sure people would be as forgiving of Nirvana going off the air for extended periods of time.
On that point, as folks here are of a more technical ilk... any suggestions for "LAMP" cloud hosting? The Nirvana backend is currently linux, lighttpd (not apache), mysql, and php (codeigniter framework).
Been looking at phpfog, dotcloud and joyent... But have also been entertaining the idea of porting the API to ruby for heroku, or possibly python for google app engine.
27 Posted by beau on 04 Feb, 2011 11:12 PM
I have experience with Linode (love them, I have an Ubuntu 10.10
instance there right now) and have heard great things about Amazon's
cloud hosting service (EC2):
http://aws.amazon.com/ec2/
28 Posted by beau on 04 Feb, 2011 11:14 PM
Also take a look at Cloudkick for a good example of monitoring/management:
https://www.cloudkick.com/
29 Posted by scottschulthess on 05 Feb, 2011 11:04 PM
I love Heroku, does "moving the api to ruby" mean porting your entire backend? I know amazon and linode are popular as well
Support Staff 30 Posted by Elbert McLaughlin on 06 Feb, 2011 10:05 PM
@scottschulthess - Yeah, it would mean porting the whole API to ruby. It wouldn't be toooo arduous (in theory) as 95% of nirvana 2 is client-side javascript, which can point to an API running on any stack. But as always, the devil is in the details, and it would be our first foray into production level ruby. Hmpfh.
31 Posted by Quick on 17 Feb, 2011 11:48 AM
Hey guys, I'm itching to see this API it's the one feature I'm really waiting for before commiting 100% to nirvana and ditching my other GTD apps :) I hope you can make it happen soon!