Drupal

The push for open data from government has been gaining momentum over the last few years. This has taken concrete shape in the form of data being released in more usable formats, including websites that are especially designed to release general data and educational data at the federal level. Some local governments are following suit.

As more datasets become accessible - and this includes comparable datasets released over multiple years - we're interested in helping these datasets bridge the chasm between data and information. It's one thing to be able to map crime incidents in a specific area between two dates. It's another thing to be able to look at some performance indicators of schools within these same areas. It's still another to be able to look at a low performing school and high performing school, and compare student to teacher ratios, the percentage of students eligible for free and reduced lunch, the number of crime reports in the neighborhood, the median income, the number of liquor licenses, and the percentage of owners to renters. By leveraging open datasets, we will enable more precise questions about how people learn, and the factors that affect learning.

In the upcoming days, weeks, and months, we will begin to document and release the beginnings of a toolkit designed to simplify building and maintaining web sites that use and display open data as it relates to education and learning. Drupal already has much of the functionality in place to support using open data, and the Drupal community can play a leading role in pushing the use of open data.

Our initial three targets include:

• Datasets: we will identify existing datasets, and document any needed steps required to prepare them for more widespread use.
• Map tiles: much of this data can be represented geographically, at the country level, the state level, or the local level. We will identify existing map data that can be used when developing location-based visualizations, and document the process of creating new map tiles. Any new map tiles we create will be released out under a Creative Commons license.
• Search and Visualization techniques: As we develop tools to import and manipulate datasets, we will need to search and filter this information, and make it comprehensible. This will likely result in new code that integrates various pre-existing open source visualization tools with the Views module. Any new code we develop will be released back onto Drupal.org

We hope that the process of looking at information related to learning will help people more effectively discover, understand, and explain the connections between student learning, school performance, the role of the teacher, and the relationship between all of these factors and the socioeconomic milieu in which they occur.

We also want more people to get involved. Toward that end, we have created a working group to discuss strategies and approaches for using open data - if you want to get started working in the space, join the group. Many hands make light work, and much of this work requires research and familiarity with how educational data is structured, as opposed to any hardcore coding or site building expertise.

Within the next few days, I'll have another post out that defines what we see as a starting point for the toolkit, and identify some of the tools that already exist that move us toward that point.

The combination of Mailhandler and MIME Router allows you to set up your site to take posts via email, and to route attachments into filefields.

MIME Router integrates cleanly with Filefield, Imagefield and SWF Tools. File paths can be set via the Token module. To use MIME Router, just upload it into your modules directory and enable it; the module uses the allowed filetypes in Filefield's user interface. Aside from adding fields to your content types, no additional config is needed to use MIME Router.

Using this setup, images, video, and audio can be sent to your site via email. This also supports posting to the site from handheld devices that support emailing files. So, if you have ever been in a situation where you used a handheld device to take a picture, video, or audio recording, and wanted to post it to your own (or your organization's) web presence, this functionality is for you. This functionality can also be used to support eyewitness information on local news sites, real-time reporting for school newspapers, and other situations where you want to get fresh information live quickly.

The MIME Router module was written and released as part of our work for the Knight Drupal Initiative.

Notes

Mailhandler settings

• Defaults: admin/settings/mailhandler
• Adding mailboxes: admin/content/mailhandler

MIME Router config

• Nothing!: it uses CCK/Filefield admin settings. Nothing to see here.

Other modules used in the screencast

• CCK
• Filefield
• Lightbox2
• Imagefield
• Imagecache and Image API
• Token (used to create dynamic paths for uploaded files)
• SWF Tools
• Mimedetect

Today at Open Source Bridge, Amye Scavarda and I co-presented a session on Thinking Like a Pirate (aka creating solid requirements docs, and how requirements docs help projects succeed).

Our slides are available here:

(As an aside, for those of you who don't know about Open Source Bridge, I'm glad you asked! It's an awesome, open source focused conference that occurs yearly in Portland, Oregon. Read more on their site.)

In our session, we fielded a question about some of the specific tools we use to build requirements docs. One of the things we use is an intake survey focused on design-related topics, and a participant asked if we'd be willing to share it out, to which I replied, "Sure!"

Which in turn meant I needed to make it pretty.

Intake surveys are incredibly useful tools. We generally use them as a way to start communication rolling on a project, and as a way of accomplishing some or all of the following goals:

• Learn about the organization's design aesthetic and needs;
• Begin to define a common vocabulary for web design and development;
• Start the conversations about the relationship between design, site architecture, and user experience;
• Begin to create the relationship between the communication leads for both stakeholders;
• Get a sense of the level of research that has gone into the project prior to beginning development; and
• Get a sense of how the organization responds to requests for information.

Below, I have included the general questions and structure of our intake survey. We vary the questions depending on the nature of the project; for example, if there is no desire to incorporate content rating, we don't ask the client to look at sites that feature content rating.

As many of you may have heard, Ning recently eliminated its free networks and laid off 40% of its staff. Currently, Ning supports exporting users; for those of you interested in migrating your community members from Ning into Drupal, you have two well developed options at your disposal:

• User Import: http://drupal.org/project/user_import
• Feeds: http://drupal.org/project/feeds

If Ning ever opened up their APIs to allow content export, then importing the content would be equally straightforward. Both the Feeds module and the Node Import module would get the job done.

Another option for data import is a combination of the Table Wizard and the Migrate module.

If you are launching a Ning site now, setting up a Drupal site to import that content via RSS using the Feeds module wouldn't be a bad idea. Should Ning go out of business, or change their terms again, or if your organization makes the decision to take control of your web presence, your data (and the intellectual capital of your community) won't be in the hands of a content silo.

Whitehouse.gov is now live running Drupal. It's great to see yet another example high-profile Drupal site go live. I'm curious to see how this story unfolds over the next few days.

I haven't spent much time going through the site, but I'm glad to see their implementation of Solr for the search features. Transparency comes easier when you can find what you are looking for.

I was also glad to see that they have implemented RDFa. I'd love to see similar steps taken on other government sites to increase transparency and interoperability.

The Washington Post has an article about the launch. It's worth reading for no other reason than to see how it sounds when a non-technical person attempts to cover technical issues. For what it's worth, this would have been a great time to use link journalism -- they could have collected up links to people who actually understood the significance of the story, and who were capable of articulating the narrative.

Update: The Personal Democracy Forum has a great post on this. Among other details, it credits Acquia and Phase2 with working on the site. Congratulations to all involved.