Amazon S3 Storage Manager for KnowledgeTree
06 Mar 2007
Over the weekend, I did the first PHP programming I've done this year, on a whim project to write a Amazon S3 storage manager for KnowledgeTree.
In KnowledgeTree, a storage manager describes where to and how to store and otherwise manage the files attached to documents in KT. S3 is a remote storage web service from Amazon, allowing reliable, scalable, and net-speed-quick storage for data. Most of the code ended up being around configuration - particularly in terms of making it easy to check whether the S3 storage manager is properly configured.
This starts with a status dashlet (a dashboard "portlet" - a little bundle of information on the "front page"), which makes it hard to miss if you haven't configured things yet:

(It also tells you if you haven't told KnowledgeTree to use the S3 storage manager.)
The S3 Storage adds an administration page, allowing through-the-web configuration of the plugin. One of the things I'm sorry I never got around to was getting configuration in KT pushed more into the database - the constant push for "cool" features meant the basics that affect actual users never got any time. Before configuration, the page looks like this:

The "No" next to testing reminds administrators whether they've done a test of the settings they've entered. The "Test now" makes it easy to see if the settings are correct without trying to add a document.
The Amazon Web Services settings are quite boring, since they can't be detected. But, for the S3 configuration, we can query the available storage buckets and present the information to the user:

Once you've set it up, it's actually quite sad how it "just works". There's no difference in terms of using KnowledgeTree to add, bulk add, or delete documents (except that it's a bit slower for those poor South Africans with not-so-wonderful connectivity). Downloading is fast, since the files are cached locally.
The entire storage manager is only 250 lines of code, most of which is caused by the immaturity in the storage manager framework, because pretty much the only non-standard storage manager that we wrote was a in-database storage manager (which was quite slow, and thus was forgotten). The admin page, status dashlet, and so forth come to 500 lines of code. You can download the plugin in ZIP and tar.gz formats.
7 Responses
trans_lux — October 09, 2007 at 01:44 AM.
It works as advertised. One ? I have is it possible to import the documents from S3 into another instance of KT say for disaster recover?
Neil Blakey-Milner — October 09, 2007 at 08:01 AM.
Since Storage Managers hear from KT whenever the location of a document changes in the KT repository, it would be possible to write an additional file into S3 with the latest location of the document, and thus create an "import manager" for the files in S3 easily enough.
trans_lux — October 11, 2007 at 05:56 PM.
One thing I did run into was on a manual install I get "Amazon Web Services settings test failed: AWS authentication requires a valid Date or x-amz-date header" This box has the correct time/date but is running PHP5. I know KT does not yest support PHP5 so do you think this is the issue.
Anyhow great work!
I love all things s3!
trans_lux — October 27, 2007 at 02:44 PM.
Nothing shows up on the dashboard or under document storage.
Any ideas on how to resolve?
trans_lux — November 05, 2007 at 06:46 PM.
Chris Stratton — May 05, 2008 at 10:46 AM.
I am not sure if you are interested in doing any freelance KT modification work, but if you are could you contact me. on the supplied email address
Claudio Cuestas — August 05, 2008 at 08:57 PM.
I just downloaded the plugin from here and about to setting things up..
thanks
Claudio
Have your say