My apologies for all the old junk that got re-syndicated last night. One of my gripes with modern syndication technology is how easy it is to accidentally have all your old posts go out again. Lame! I'm not sure which part of the stack is to blame, so I just blame all of it. Of course, I'm sure *I'm* to blame somehow, but I'm actually not sure what I did to cause that to happen.

Anyway, in other news... Account Manager got rebooted today as a Firefox project. Exciting stuff, though we have a lot to do still! But it's really nice to start to see all the pieces fit together, like how this might interact with the Contacts add-on Mike is hacking on, or other fun stuff we have in the works.

Also, today I saw Jono drink some OJ with added MSG after he ate some miracle fruit (!). I have pictures, maybe I'll post them later ;-)

In other *other* news, I've been having a ton of fun with my new camera, you can see some fun shots (including Labs craziness) at my flickr page.

For a while now Weave has had an identity component that we've kept exclusive to Weave. These past couple of weeks we developed a prototype that uses that identity to log you into OpenID-enabled sites. Not only that, but over the weekend Myk integrated the saved passwords and Weave logins into a single UI off the location bar.  What's more, this new UI also lets you automatically sign in to sites!

I think it's a super compelling demo, I'm using it now and loving it.  To check it out, download the latest Weave snapshot here.

Or if you want to read more about it (or watch it on video!), head over to the labs blog here.

Happy New Year!

Good luck in 2009, everyone :-)

2008 was a year of change and experimentation for Weave.  During the first half of the year our focus was to increase our breadth: we implemented synchronization of cookies, saved forms, input history and passwords (in addition to bookmarks and history, which we implemented at the end of 2007).  We also worked on adding end-to-end public-key encryption so we could support 3rd party access.  We had small experimental OAuth support and web frontends to use it.  We implemented a variety of client technologies to help us: logging and trampolining libraries, UI notification boxes, and more.  Some of these are actually being adopted by other projects, which is incredibly gratifying.

The second half of 2008 we took a step back and evaluated how to go forward.  It was clear that, though we had gone far, we would face some significant scalability limitations: both in scaling the service to millions of users, and in scaling Weave down so it could fit in a mobile device.  Both were things that we strongly desired to see Weave do, so with Toby and Jono's help we designed a brand new server API and a new user experience specifically for Fennec.  Much of the client (and all of the server) had to be rewritten to make this possible, and we're extremely proud to have quietly released a preview last week with Fennec support (to go with the Fennec alpha 2 release), which a few thousand people are already using.  I was able to synchronize more than 700 bookmarks to my N810 running Fennec and immediately search them using Fennec's awesomebar.  Talk about exciting!  I'm happy we were able to mark the end of 2008 on such a high note.

But of course, that is just the beginning.  Where will Weave take us in 2009?  In the first In the first few months we plan to quickly release several milestones, the first one scheduled for mid-January, and work towards syncing up more closely with the Fennec release schedule.  We will have support for bookmarks, history and passwords for Fennec, possibly others as well.  And we'll work to have a smooth and tightly integrated user experience with Fennec, working invisibly in the background as much as possible to allow you to take your Web with you wherever you go.

In the following months, we have a variety of projects we may focus on.  We have a big interest in secure 3rd party access to user data, to unleash the creativity of web developers out there to work with user data without having to collect it manually.  We're also interested in syncing other data types, including microformats and other structured web content.

As has been previously reported before, Mozilla Labs will be making an appearance in London, Berlin, and Barcelona next week.  We would really like to connect with our community there, so take a look at our schedule below and join us :-)

I'm really excited about the whole week.  We have some pretty cool stuff planned out, particularly for the Berlin Hack Days where we'll have 3 days of low-key events.  We'll be showing off the latest from Labs, and we are planning workshops and contests (we'll have goodies and prizes!).

PS: I know my last post promised I'd talk about the upcoming Weave architecture changes.  All I can say is... stay tuned for my next post :)

Monkey Goose!

It's been a while since we started the Weave project, so I thought this might be a good time to take a step back and evaluate how things have gone so far, so we can better understand where to go from here.

Perhaps I should start with a bit of background. Weave started from two separate efforts: One was part of the Places work for Firefox 3 where the main goal was to enable users to synchronize their bookmarks and history among their Firefox installs. The other was a project to integrate services into the browser in innovative ways. These two efforts joined to create a unified vision where data synchronization was merely one potential service we could offer, as well as a foundation for other services to be built upon (by both Mozilla and 3rd parties).

Integrating services into the browser is not a radically new concept, but it's not something Mozilla has generally done a lot of before. Search, software update, anti-phishing/malware, and (recently) the add-ons manager are the biggest services that are currently integrated into Firefox.  But it definitely felt like we were treading new ground when we started thinking about building an extensible architecture that could provide coherency to what we've already done in terms of services integration, and at the same time lay a foundation for evolving the browsing experience even further.  Putting user data in the cloud is really just the first step. We laid out some basic principles and started exploring our options.

We decided fairly early on that if we could use an existing server piece it would make our lives easier. If it was close to plain-old-http then even better, since we already have expertise in-house for scaling web sites (Mozilla web sites are among the most popular sites on the web!). So WebDAV was a perfect choice: it freed us from having to worry about the server, and allowed us to focus our small team on the client alone.  But perhaps more importantly, it makes it very easy for other people to set up and maintain their own servers.

Fast-forward to today, and we've accomplished quite a bit. Looking at the Weave client today, it's amazing how much it has grown (it clocks in at around 18 thousand lines of js and xul), and it can do quite a bit! It has even spawned some mini-projects, some of which have been reused in other extensions (e.g., the logging framework, trampolining code, etc).

But the point of this post is not to pat ourselves in the back. We think Weave still has a long way to go, and the real questions are: What do we need to do to get closer to our vision? What are the main problem areas right now? And, do we need to revisit any old decisions? Now that we've been running this experiment for a while, I think we can begin answering these questions.

Most current problems with Weave are related to speed and reliability. In part this is because we didn't know how well things would scale--that was part of the experiment.  But the biggest reason is really that we didn't focus on performance.  Instead, we chose to focus on getting basic functionality in place.  We think we have that now, so we're making the next iteration about enhancing that basic functionality and allowing it to scale.

Weave Sync is designed to take a very holistic view of the data and decide what to do. This holistic approach does have some good points, but performance at scale is not its strong suit. Possibly something we could revisit, and in fact we already have to some extent by looking at alternative 'sync engines' which don't necessarily use the same sync algorithm.

As for reliability, it is a complex problem, but bugs aside the biggest causes of unreliable behavior have been keeping the servers running smoothly, and complexity in the client from using simple file storage (WebDAV) as the server API. The former is mostly a function of the available hardware/bandwidth and our skill at tuning it. I think we've actually gotten pretty good at it, but we're still running into some hardware limitations (disk i/o), and we keep hitting some disk space issues as well, in part because our storage model is wasteful: no compression, change history is kept forever on the server.

The problem about extra client-side complexity is perhaps a bigger problem, and it's actually partly to blame for the storage issues.  By simplifying the way the client stores data on the server we can simplify code on the client while at the same time making it easier to scale server storage.  Thus, we've chosen to attack our problems from this angle first.

What does that mean, exactly? Stay tuned for my next post :-)

I really don't blog enough. Here goes:

<thunder> hmm I should get with the reading of the blogs
<thunder> dria: I tried out feedly and I was really impressed for a little while
<thunder> but I'm not sure I can stick with it
<dria> thunder: how come?
<thunder> for now, anyway
<thunder> hm
<thunder> so, I really love the magazine idea
<thunder> but it doesn't paginate
<thunder> so I end up going to the river view, or just clicking on each blog on the right
<thunder> to read everything / mark everything as read
<dria> thunder: ah.  yeah i just use "river" view...magazine feels too random.
<thunder> right, it does
<thunder> but the river view is not so different from my previous reading habits
<thunder> except it pulls the images out, that is cool
<thunder> I think the magazine view is really where it's at, but it needs some tweaks
<dria> yeah.  i've had to reorganize my feeds to work with feedly.  the old system just didn't work
<thunder> I had an idea of sorts I discussed with myk when talking about snowl
<thunder> I think what I want is the idea of an 'edition'
<thunder> which is basically a date range
<thunder> once you restrict the date range to a day (or 1/2 day if you have a lot of blogs)
<thunder> you can start to do some pretty cool stuff, like the magazine view
<thunder> the good thing is, you can then paginate
<thunder> so if I browse to yesterday's edition of my feed I don't need to mark everything as read
<thunder> because things I don't read will not clutter later editions
<thunder> and within that edition you can use page numbers to move around
<thunder> much like a newspaper
<thunder> (you can also have sections within each edition)

Also, have you read Jono's latest post? Awesome.

Yoshino-san is working on a fantastic web application, seeing a demo of it today almost made me cry from happiness and anticipation.

Please ask him about it, let's make sure he continues to work on it ;-)  I for one will be a regular user if he releases it, and I suspect many of you would be as well!

Koheiさん, 頑張って！