Blog

Do It With Drupal: Configuring a Kick-Ass WYSIWYG Editor

  • To make it easier for people to format text without knowing HTML, you could do filters, and/or editors
  • Filters change stuff - people type things, filter changes that on output
  • A ton of modules that provide filters in addition to the core
  • Filter only makes its transformations on output: this means that whatever your user puts in the text box, it's stored right the way they did it, and you can transform it any way you want after the fact
    • Transformation happens after it gets stored, so you don't lose the data the person input; also, what if you want to filter it a different way later?
    • Change the filter settings, you're not changing your data
  • "Input formats" become "text formats" in Drupal 7
    • A format is a way to group a bunch of filters together
    • This is an important distinction (format = collection of filters) when reading documentation
    • Drupal Core comes with some formats: filtered html (default), full html
    • Filtered html helps with security, strips things down - otherwise, you can put javascript in the text box (dangerous)
    • Filtered html works as a whitelist
    • Don't get frustrated and switch to "full html" - security nightmare; people do Google searches for text that allows full html for comments for anonymous users
    • Core also offers PHP format - you have to turn on PHP filter module first (BIG DANGER)
    • Filtered HTML: URL filter, HTML filter, line break converter, HTML corrector
    • Full HTML has the same, but removes the whitelist

An example

  • Even when you're using filtered HTML, if you put a script tag in there, it's still saved in the database just like you typed it
  • The database also specifies what format to use
  • URL filter, HTML filter, line break converter, HTML corrector fire in that order - order can be important (one filter can "clobber" another)
  • Filtered HTML also strips attribute tags - sorry

Markup filters

  • Markdown, Textile, BBCode, Wiki Markups (Mediawiki, TikiWiki, etc)
  • Other cool filters: code filter/GeSHi, SpamSpan (to avoid e-mail scraping), Paging, Pirate (Talk Like A Pirate Day), Typogrify (turns two dashes into an em-dash, etc)
  • It depends on your user: if your users are used to Wikipedia and they want Drupal to be a wiki, you can do that
  • Sometimes you don't need a WYSIWYG - a filter might get you what you need

WYSIWYG: just gimme what I want

  • There are also HTML helpers - it'll insert the HTML tags for you, that might be a good compromise for some people (BU Editor is the most popular one, probably going onto drupal.org)
  • Back in the day: tinymce (awesome cool javascript thing)
  • For a while, tinymce was what everyone used
  • Also out there: fckeditor (name is changing to "ckeditor"; fck are the guy's initials, non-native speaker of English...)
  • Huge threads on whether tinymce or fckeditor is better - comes down to preferences
  • New: WYSIWYG API (actually just called WYSIWYG)
    • Central place for you to plug in whatever editor you want
    • Flexibility to be able to have any editor, and more than one at a time - say when you want to use what
  • Dealing with images (arrrrrghhh!!!)
    • Things are getting better
    • imagefield - can actually stick them in the body text
    • imce - image browser integration

Questions

  • How to strip out content from Word
    • People who want to just copy and paste stuff from Word - some of the WYSIWYG have a "strip Word crap out" button
    • Alas, there's no module that does it right (been there, tried it all)

Demo time

  • "Input formats" under "Admin" - filters are part of formats, so go looking there
    • If you go to modify a format, you can turn filters on/off (image resize filter, markdown, others can be added)
    • Every filter can have its own configuration - things like maximum link text length
    • "Rearrange" lets you specify what order the filters fire
    • Check out the readme for filters; many of them will tell you what order they should be in
    • Markdown should be before HTML for security, to strip out anything weird that Markup generates that's not on your whitelist

Tags: 

Do It With Drupal: Geolocation

  • You can use full html to input a map straight from Google
  • Geo, Geocode, OpenLayers modules
  • Standards compliance
  • Example functions: within, touches, crosses - also, distance, area, perimeter, and others!
  • PostGIS is a common way of doing it, but you can also "do it in Drupal"
  • User friendliness: you shouldn't have to be a cartographer!
  • "Geo-spacial data sets", shape files, projections, coordinates - this stuff can get messy fast

Collecting data

  • Run everything through CCK
  • Geo Field
  • Geocode

Lines

  • Can import a line (big mess of points)
  • OpenLayers - takes your data and turns it into awesome, pretty, rendered maps

Data

  • Stuff you can geocode: image fields (exif), file fields (gpx track logs)
  • Demo site: http://geoblog.geojune.org/
  • Collecting tracks: dedicated GPS, gaiagps.com (iPhone), emacberry.com/gpslogger.html (Blackberry)
  • Data sources: data.gov, zillow.com, DataFinder, etc.

Questions

  • Compatibility with other modules? - A lot of work is monolithic, people don't work together; there may be a CCK thing for Mapstraction
  • GMapEasy module? No idea...
  • Before, people were doing Location, but that was overwhelming; then people did a simple lat/lon field; the views code was starting to get messy
  • You shouldn't write a module that has to hunt around and look for things - try to get all the people who work on geo stuff to express their data in a consistent way (rather than making a module check all the possible ways to store data)
  • How many ways can you pull in data? - Geocode module is open, just tell it what kind of data you want to give it ("I'm going to give you an IP address"); no support for adding Geo stuff to users, but it's an easy API change
  • In Drupal 7, when you can add CCK fields to users, the geo field becomes available
  • Any way to define/create your own graphics? Mapping Middle Earth? - No technical limitation, Geo module doesn't care what you put in there; Geo just stores points/lines/polygons/3D, etc.
  • OpenLayers - finding graphical tile sets for your map of Middle Earth, that's a whole secondary technical problem, even if you've got the right coordinates in there; there's pieces on the display side that need to be there
  • There's a filter for "less than/equal to X miles of this point" in Views - this saves you from math

Modules

  • Geo - storing your data
  • Gathering your data - geocode, postal
  • Showing it - OpenLayers

Tags: 

Do It With Drupal: The Power of Features

See also Features on drupal.org

  • Jeff Miccolis & Eric Gundersen - Development Seed, building a lot of products (things like Open Atrium)
  • Drupal is very configurable - but that's also a weakness: no distinction between what's configuration (views settings) and what's content
  • Workflow problem: when you build a site, you build in a dev environment, but client/boss wants to see what it looks like before it goes live
    • So, you stage it somewhere, then move it over
    • Development: where the action happens (possibly your laptop)
    • Staging: where it's reviewed (much closer to where it's going to live)
    • Production: where it's live. (developing on the live site is always a bad idea)
  • Three people working on a project that needs to go live
    • Musician, developer, themer
    • Round 1 goes great - everyone works together and the site goes live
    • Round two is a PITA: new views build on dev, rebuild on staging, rebuild on dev, rebuild on staging, over and over, rebuild on prod
    • Extensive note taking, prone to human error, loads of repeated tasks
  • The solution? Make a distinction between config and content - views and settings are heavily and clearly distinguished from the actual content - then write this configuration to code and get it out of the database
  • You can do version control with your config - this lets you track changes
  • Node types, CCK fields, menu, blocks, views - these are config
  • You can say "these components taken together define a feature" - something the site does
  • "Features" module - Feature = Drupal parts that do something specific (Views, ImageCache presets, content types, fields, etc.)
  • Features = Drupal module that allows for the capture of configuration into code
  • (Sorry about the name; the Feature module makes Feature modules which have things)
  • Feature modules have Core exportables: content types, permissions, input filters, menu items
  • Contrib support: contexts, views, ImageCache, Ctools (panels, feeds, etc.)
  • Features is a system to capture the various components that describe how your site behaves
  • Features should be used throughout the development process - you can take a live site and capture existing features, but it requires you to change your thinking about how users interact with the site
  • Concepting what's part of which feature, what's shared, etc. gives you stronger features

Making Features

  • Create a Feature: you can add components, cycle through various elements, clickthe ones you want in your module
  • Features come as a nice tarball - turn it on in your website, you get all the stuff that comes with it
  • But then people start changing the view - you can see the status in the Features module (has it been changed?)
  • If something has been changed, it'll show you what
  • "Recreate" button will give you another tarball, with the current state of things

Create, Update, Revert

  • Drush commands - features, features export, features update, features revert
  • Views changes are made only once, each change has a commit log, if you check it into SVN like you should
  • If you move your development to a real dev environment, and leave the staging site as a staging site (that you can show clients, etc without worrying it broke in the last five minutes) this is good

Distributing Features

  • Are your features appropriate for drupal.org?
  • Is the configuration an IP issue?
  • How can I get that nifty update status thing behind the firewall?
  • If you can't/don't want to send it to drupal.org, but want to manage it internally over time: Features server
  • code.developmentseed.org/featureserver
  • Create projects, make new releases, subscribe to updates, etc
  • For automatic packaging, try the Project module
  • Feature server is much simpler, lets you get off the ground fast
  • Based on implicit standards: update status xml, exportables, drush make

Tags: 

Do It With Drupal: New York Senate

Background

  • Transforming an anachronistic organization with Drupal
  • In control of Republican party for 44 years
  • Never had a CIO before January 2009 - focused on internal enterprise IT before
  • People were cutting out and pasting articles from papers, scanning them, printing them, and distributing these reams of paper to offices every day - 1.5 million/year
  • CRM (constituent relationship management) - command-line type system
  • Intranet 1.0 - publishing info, no collaboration
  • Desktop PCs
  • Email 1.0 - intranet only, can't work from home
  • Managing our own data center - not a core competency, but we do a reasonable job

NYSenate CIO Mission

  • Transparency
  • Efficiency - more effective, less cost
  • Participation - give people a participatory role in government
  • Modeling 'best tech practices' for legislative bodies
  • Organize/share data internally/externally, improve internal/external communications

Site dissection

  • No staff with web development experience in January; started out w/ consulting firm
  • Built by April, launched in May
  • Had to train hundreds of staff people to use it as content creators
  • RSS feeds, Twitter, Facebook
  • Popular/e-mailed/commented content, events, press releases/blogs/news clips
  • Almost 100 sites in one: 62 mini-sites for senators, 40-ish mini-sites for committees, issues/initiatives, legislation, open senate, about, photos & videos, newsroom
  • Previously, used proprietary CMS and external vendor - one party got better sites than the other, even with tax payer dollars covering everything
  • Senator directory - shows RSS/Twitter/Facebook (when available - been actively promoting this)
  • Senator pages: they stand on their own, all the info about the senator, he can post news releases/blog, news clips related to him, videos, RSS/Twitter/Facebook
  • Senators can create stories with visuals for their pages
  • Committees - each has its own stand-alone mini-site, with chairs, sign-up for newsletters, updates, video archive of meetings (will be live streams in January)
  • Submitting testimony on-line available in January
  • Issues & initiatives - marriage equality (aggregated all content from site), PSA (information about the census)
  • OpenLegislation: information should be freely available, searchable, sortable, permalinks
  • Open Senate initiative: OpenData (administrative info, how much who gets paid, what gets spent on what, etc.)
  • Data available in different formats - PDF, CSV, TXT, XLS, DOC
  • Contact forms for senators individually and for the site in general (press inquiry, webmaster)
  • Photos and videos - recording and, soon, livestreaming everything
  • Also available on YouTube; audio available on iTunes
  • Working on adding automated transcription
  • Blogger who works in the "newsroom" to create web-friendly content/press releases for the site

Modules

  • 131 modules + core required: activism, petition, administration, gmap/location modules, content templates, interrelated date & calendar, imageAPI/imagecache, and more!
  • Views: home page image carousel, event calendars, video/photo galleries, press releases, petitions, senators' pages
  • CCK: constituent stories, senate districts, events, expenditure reports, photos, polls, press releases, video, senator, committee)
  • 19 custom modules - custom views/blocks for the most part, permissioning system for Office and Web Editors
  • Upcoming: distributed authentication, ideas crowdsourcing, unified commenting
  • Working on implementing SOLR search - Acquia is now hosting our site as of today, we've so far been using native Drupal search
  • Embedded Media Field for video

Integration with other applications, social web

  • 15,000 viewers on livestream.com for marriage equality debate
  • Social bookmarking for all content on the site
  • Some senators are using Facebook well and having open discussions with their constituents
  • nysenate.gov was re-branding, now we use "nysenate" for everything
  • API so developers can take any of our open data and do things with it
  • Haven't made a final call about whether to keep using Discuss (external product) for commenting, or use Drupal's native commenting (there's a lot of configuring to do to get the seamless experience we want)
  • Sign up for updates about anything on the website; integrating w/ Bronto for e-mail blasts
  • Voting content up and down - needs to be elegant and incredibly easy, using a 3rd party solution right now and themed it like the main site

Everything else

  • New hosting - don't have the resources to host something like this; now moved to Acquia
  • New domain name - wanted .gov to force the issue of what you can/can't say (previously, it'd been used to say partisan, sometimes nasty things)
  • New policies (content creation, copyright, privacy, TOS, release of data, permissions)
  • New processes (requirements gathering, quality assurance - people who had previously done phone service or legacy systems, content creation workflows)
  • New talent (previously didn't have any web developers in-house, consulting contracts, staff)
  • New tools (videoconferencing, IRC Chat, Central Desktop- lightweight project management, Redmine- bug/feature tracking, ticketing tasks)
  • New training materials
  • New communications/PR

Guidelines & miscellanea

  • No political or campaign information - conveniently, with .gov we're not allowed to
  • Copyright policy - states can assert copyright if they want, but we went for CC BY-NC-ND for most things
  • Privacy policy - mirrored White House
  • Terms of participation - also mirrored White House
  • Post all code to Github
  • Use Daylife.com for replacement to paper clipping system
  • Hope that other legislative bodies will be able to reuse code
  • Had an Unconference (CapitolCamp) to hear what people think - some people were excited to pitch in, do things with API

Questions & feedback

  • Node Bulk Operations could be helpful
  • Had to take screenshots for a while to allow very non-tech-savvy senior people to see private things without the risk of them doing anything wrong with it (finding a better way for this)
  • Feedback from senators has been all over the map - actually the inverse of expected, where more Republicans were early adopters even when they weren't saying nice things about it in public
  • More Republicans were effective using Twitter and Facebook, more internally organized to identify opportunities and make the most of them collectively
  • Senators are learning that by making content easy for others to see and share, related content gets more views too
  • Google Analytics stats available for all senators available; special reports around particular events
  • 1.5 mil page views a month, on a big day, 50,000 unique views (marriage equality)
  • 40-50 comments on a hot bill
  • Not massive, shouldn't cause major performance headaches, but we had to do this in such a rush that we have a lot of refactoring to do to make sure it holds up okay under stress
  • If there's something broken, blogs publish screenshots - we have to be very vigilant
  • Want to make custom modules available; just haven't had the bandwidth, just have a code drop on github for now
  • Building relationships with CIOs of various state agencies - some of them have a lot more developers
  • PDFs have been the traditional publication format, including scanned documents; we've maintained that format for most data to accommodate the "I want to download and print" crowd - only last week got wifi in capitol building
  • For born-digital content, making it available as feeds in ways that will make it easier for people to use
  • More and more federal work being done in Drupal (whitehouse.gov); a couple state entities have put up rudimentary sites (liquor authority for state of New York)
  • Contacted mostly about policy issues for other states - comment moderating, copyright
  • Big national open data initiatives - community of practice around government transparency
  • Haven't sat down with whitehouse.gov Drupal developers to talk about roadmaps yet - we feel overwhelmedly busy right now
  • Third party to compare roadmaps, sort out implications for working together? It's a major undertaking
  • Sunlight foundation - encourages getting data out in mashable form; they give us feedback
  • Some senators have gamed the system by getting people to e-mail things they post so it gets on the "most e-mailed" list - this upsets other senators

@ahoppin
@NYSenateCIO
NYSenate.gov/department/cio
Hoppin - at - Senate.State.NY.US

Tags: 

Pages

Subscribe to Blog