Maitri's VatulBlog

You read yesterday’s VatulBlog post on natural disasters and the benefits of educating yourself in this time of information plenty.  Where do you start, however, when search engine output has a rather low signal-to-noise ratio?

1) A list of Google’s or Ask’s search terms shows many internet users cannot pose a question to save their lives. Use these 20 Tips For More Effective Google Searches to avoid being overwhelmed by search results.  Become one with Boolean.

2) There are many free science search tools out there.  Here is a review of the prominent ones.  Search early, search often.

3) Ask a librarian.  There exist these humans, many Library & Information Science graduates and other smart people, who work at libraries who can help you hone your search skills as well as find answers to your questions.  They also do funny dances with library carts, I hear, which is always worth the time spent at your local library.  Jokes aside, considering the number of people who don’t possess home computers and computer & online literacy, well-funded libraries and intelligent, helpful librarians are crucial to the future of information facilitation.  Also consider publishing your search results and analyses as a blog post to help others who may have the same question.

4) For humanity’s sake, lower the noise or help verify data.

Blair pointed me to The Economist’s The Data Deluge which simultaneously celebrates and bemoans the exponential increase in data, be it photographic, tabular, scientific or vital-statistical in nature, and how corporations are only beginning to find value in separating the wheat from the chaff.  Violation of privacy aside, the greatest threat to signal quality is data hoarding (especially by individuals, taxpayer-funded organizations and publishing houses) and a large amount of noise in the system.  Those who do put information out there don’t (want to) curate it, which includes ensuring accuracy and constant updates.  In a comment on the aforementioned Economist article, a D. Sherman says it best:

… the vast majority of of “noise” in databases is simply bad data, duly entered and propagated … We put a great deal of effort into collecting more and more data, but comparatively little into weeding out bad data. This implies that the sign[al]-to-noise ratio is only going to deteriorate. Part of the reason for that is that the incentives all favor collecting more data, but do not adequately penalize bad data.

… Members of the database resistance movement who are willing to risk more radical means of spoofing more important databases can readily imagine even more creative (though less legal) methods. When RFID tags and image-recognition tracking of people and vehicles becomes more common, the opportunities and means for injecting noise into the system will multiply exponentially.

The solution is a selective and sophisticated willingness to part with data depending on what the data involves.  Not divulging personal information and data hoarding are two different things.  While I value my personal privacy and often purposefully give wrong phone numbers and zip codes to websites and salespeople, I will readily part with any scientific, technological or social information that is not classified explicitly as proprietary.  Google and Bing are not entitled to my street address but they are to my paper on transtensional folding because Americans paid for it and not Elsevier or Springer.

Another sure-fire way to lower noise is to cut back on content replication.  A mature search engine will a) enforce canonical URLs and b) take a scythe to unabashed content scrapers who enjoy high search engine rankings.  I’ve lost track of the number of MY blog posts that show up on others’ sites as higher-ranked search hits.  (Just because my content is published under a CC-BY-NC-SA license does not mean you are entitled to pilfer it and ignore the non-commercial and share-alike components of the license.)  Also, take-down notices only work if you can find a responsive human on the other end to respond, and who has time for that?  There has to be an easier way to punish websites and aggregators for outright plagiarism and internet abuse.

Speaking of content replication, here’s an interesting “what-if” article on information copying and machine sentience.  I guess there is one benefit to a low signal-to-noise ratio in the network: really dumb Artificial Intelligence, should it emerge.

Needless to say, you’ve all heard about Apple’s iPad by now.  I’m certain Steve Jobs’s unveiling speech yesterday was more popular than Obama’s State of the Union address, judging simply from the crazy high TPM (tweets per minute) related to the new gadget’s drop.  (Disclosure: I contributed to said traffic with 9 tweets and around 10-12 replies and retweets.  Glad to have done my part.)

As a technologist and tech blogger who attends conferences regularly, I am in the market for a new portable computer that is a little lighter and faster than the existing Dell Inspiron. Size, shape, bezel and “form factor” are meaningless to me.  Following is what my portable computer has to DO, along with what the iPad does (green) and doesn’t (red) offer as a solution:

* Word-processing program for rapid note-taking, with machine on lap or standing at booths with very little counter space. $70 keyboard dock.  The awkward iPad-keyboard size ratio and keyboard’s tilt make it seem unlikely the setup will stay put on a lap.
* Wordpress post creation and editing in full visual editor.  The WordPress 2.1 app or WordPress in Safari should work pretty well for this (images may have to be resized and repositioned later, especially on photojournalism blogs).
* Occasional code testing in Python or from a terminal window An impenetrable Terminal app exists, but other than that, I haven’t tried any such thing on my iPhone.  Anyone?
* Upload photos to Flickr directly from device OR quickly connect camera/device to computer, crop/adjust/saturate and upload to Flickr or to blog post.  No built-in camera. $29 Camera Connection Kit with two dongles that plug into the keyboard dock connector; one for USB and one for SD cards.
* TweetDeck. Check.
* eReader sans DRM. iBooks is right out because it cannot read Project Gutenberg plain texts or anything other than the EPUB format. Not supporting that crap. The Stanza App will continue to read all formats, but will lack “form factor” of iBooks.
* Standalone GoogleChat. IM+ App or m.google.com
* All of these programs running simultaneously.  NOPE!

Secondary requirements:
* Ability to view videos and HTML5 content.  YouTube app for video. No Flash (no Hulu for you!) or HTML5.
*  iTunes which accesses my 8GB iPod or my 20+GB iTunes music library.  HAHAHAAAA!  Dream on!
* Not having to deal with AT&T. What are my options here?

When I mentioned some of these points to a colleague who is seriously considering buying an iPad, he said, “I really don’t think it is meant to be a note taking device or for other uses you mentioned. Those are called laptops.”  This person is also going to get the $500 base model (16GB; WiFi only, no 3G) for “casual use at home, looking at the internets, watching videos, reading books, pictures, etc.”  Another friend is going to buy it as a second home computer, while yet another will purchase it as an eBook reader with internet access.

In no way will the iPad replace your phone, MP3 player, camera and a laptop/desktop, which you will still need to make calls, listen to much of your music, capture photos and video and do any substantial work.  Moreover, as the HotHardware review says, “If we’re going to carry around something that requires a separate bag, we want it to have a real desktop and real multitasking capabilities.”  It is, however, a cool toy with which to block the television while seated on the couch, read at the cafe or restaurant during those oh-so-frequent breaks or fall asleep with.  A large-font eReader that surfs the internet and runs apps without having to run a giant OS.

Therefore, let’s not kid ourselves about the iPad as Disrupting Gamechanger That Forever Changes The Face Of Computing.  That day is not yet here.

***

In my 2009 VizWorld tech wrap-up, I wrote, “There are many more people out there who simply want access to maps, books, music, data and that is the real imperative upon revolutionary, disruptive technologies.  We cannot swallow the eReader marketing pill because it’s handed to us and, in our obsolescence-inducing plenty, unwittingly set data standards for the rest of the world. Consumers going into the second decade of the 21st century must focus on content and delivery – useful content in an accessible and understandable format on a relatively fast and ubiquitous machine – as their technology drivers. Open data, better communication and scrutinizing intent in this day and age of Twitter and other social media will make this happen.  But, so will awareness, responsibility and active participation.  In 2010, I ask us to be mindless consumers less and nurturing communities more.”

So, think critically about the social context of the iPad and read some more before you make this purchase.

Lizzy Caston and I were to write a mode d’emploi for air travel in this day and age of the ever-orange threat advisory.  A sample: Lady, please do your best not to wear four-inch-heeled slouch boots and every metal ring and bracelet in your collection before entering airport security.  The grimace on your face as you hobble about like a startled flamingo while trying to yank that thing of your foot amuses no one and only makes us standing in line behind you at 6AM want to push you down and carry on. Ok, it wasn’t going to be snide and actually more polite and helpful, honest.  Given recent explosive and “explosive” events and evolving TSA guidelines, however, Lizzy and I are going to have to sit on a few more flights, visiting a few more airports in the process, before we can pen anything useful.

Lists it is, then.  Best of decade (never mind that the new decade technically does not start until January 1, 2011) and best of year lists.  Ranking things is not my cup of tea; all of my top five movies rate about the same.  But what motivates others’ sort algorithms and makes their #1? Let’s see.  As always, please add to the discussion and feel free to list your favorite lists in the Comments section.

TECHNOLOGY – Since my Precious iPhone has not been more than arm’s length away at any given moment this year, to the point that my husband thinks I need to “tweetox,” it seems only fitting to start with Wired’s 20 Favorite iPhone Apps of 2009.  Productivity is king, followed by games, travel and hobbies.  Am I supposed to be embarrassed that I’ve downloaded only 2 of the 20 – Runkeeper and RedLaser – or proud to have gone this long without spending money on some of these not-free apps? $5 for Instapaper when I can simply Safari over to reader.google.com?!  I think not.  What are some of your favorite apps and why?

MORE TECHNOLOGY – The Real-Time Web is all that excites me in this list of 2009’s disruptive technology. Augmented Reality has potential but, in my opinion, isn’t ubiquitous enough to have made a difference yet.  Google Voice and Wave haven’t shown me their value this year, either.  What do you think? PC World’s list of the 10 disruptors of the last quarter century rings truer even today – I highly recommend this read.

MOVIES and technology – Roger Ebert is a rockstar.  Here’s a man who can find a great movie in a stinking haystack, commit to his picks and explain patiently to you why.  Ebert’s on Twitter, where he points us to all four of his Best Films of 2009 lists.

Aside from watching the Lord of the Rings trilogy, Matrix trilogy, Harry Potter saga and a handful of pop and arthouse films in the theatre this decade, I have to admit that D and I are not the best cinema-goers, preferring to watch DVDs in the comfort of our home (Netflix – now there’s a decadal gamechanger mentioned little), and even that has fallen by the wayside.  But along comes streaming video, the Creative Commons (also one of the best concepts given form in the 2000s) and the notion of simply putting your art out there, the studios be damned, and you get beautiful genius like Nina Paley’s Sita Sings The Blues, which has been around a lot longer than you think.  Whether a movie has live actors or animated ones, the most important thing about it is the story.  To paraphrase my dear, departed 3D Arts professor, George Cramer, all the visual effects dreamt of in Hollywood cannot polish a turd of a story.  This is why I am not likely to watch Avatar and recommend Monsters, Inc. instead.  Excellent story + well-animated fur = WIN.

MUSIC and technology – NPR’s The Decade’s 50 Most Important Recordings Between YouTube and downloadable MP3s, my music collection grew and grew up in leaps and bounds this decade.  Ignoring the current obsession with emo-hipster bands, pop divas and American Idol ingenues, there was some real good stuff: Radiohead’s Kid A and self-released In Rainbows, Kanye West’s College Dropout, Madeleine Peyroux’s Careless Love, The Flaming Lips’ Yoshimi Battles The Pink Robots, The New Orleans Bingo! Show’s For A Life Ever Bright, Neko Case’s Fox Confessor Brings The Flood, Sasha’s Airdrawndagger, DJ Krush’s Jaku and OutKast’s Stankonia.  Coolest music videos of the decade: Ok Go’s On Treadmills, Clint Maedgen’s It’s A Complicated Life and Empire Of The Sun’s We Are The People.  Alright, folks, tell me what I missed and why.

BOOKS and technology – Forget the Kindle and nook.  And forget those who tell you this carefully-planned obsolescence is going to change the nature of reading.  Find a light laptop and/or smartphone you’re comfortable with, do actual work with it and download books to it.  A book is not an exotic bird to be placed in the gilded cage of DRM, but something to be owned, shared and, most importantly, read many times on any platform.  I’m against the iTunes model of book consumption – fit the media to the unique delivery mechanism – and publishing companies’ constant war on the public domain.  Hooray for copyfight and folks like Cory Doctorow who have the balls to self-publish quality literature.  True defenders of freedom will enjoy and be inspired by his Little Brother.

The Times Online’s 100 Best Books of the Decade. If you’re going to read only one of them, make it Junot Diaz’s The Brief, Wondrous Life Of Oscar Wao.  What a book.  A Dungeons&Dragons-playing Dominican-American college student, the gut-punching dialogue and relationships, the history, the profanity oh the succulent profanity, the future.  What a book.  Your best read?

SPORTS – An NFL junkie, my most important moment in sports was the New Orleans Saints’ 2006 return to the Superdome after the Flood and Tom Benson’s near treachery.  Granted, it contained no triumph of athleticism, but you’d think it rates (sorry, brimmy, but the Patriots-Brady-NFL-ESPN lovefest is getting old).  Again, is there another such moment in the 00s that I should be aware of?  Please comment away.  Lastly, for the record, I am very pleased with my quarterback and Athenae’s imaginary boyfriend, Aaron Rodgers.  His stats this season show that the team made the right decision and everyone else should shut it.

TELEVISION – When Babylon 5 ended in 1998, I despaired.  This is why God invented Deadwood, Battlestar Galactica, The Wire, Rescue Me and Futurama.  After Lando Calrissian and Kosh, the outstretched arms and biting sarcasm of Bender and Tommy Gavin beckoned. After Katrina and the Flood, the confused innards of Al Swearengen’s Deadwood, Adama and Starbuck made all too much sense.  The Onion AV Club’s Best TV Series Of The 00’s nails it.

LIST TO IGNORE – Ten Stories that Changed Our Lives This Decade: #10 Katrina. #9 Brett Favre.  Delete.  Any list that places Favre, whose family was very much affected by the hurricane, over the suffering of people goes right out the window.

THE WEIRD AND CHEESEHEADY, ‘cuz that’s how we roll – 2009 in Review: Top Weird Stories From Wisconsin: “A 37-year-old Fond du Lac man went to a motel room for what he thought was going to be a romantic tryst but instead was assaulted by four women who used Krazy Glue to attach his privates to his stomach. Police say it was all part of a bizarre plot to punish him for a lover’s quadrangle gone bad.”  Hey, we gave you Ed Gein and Jeffrey Dahmer. Enough said.

ROUNDUP – I’m a sucker for New York Times graphics.  Philip Niemeyer pictures the past 10 years in a neat little 12×10 matrix.  The word “truthiness” gained popularity around the same time as Katrina/Federal Flood (they used a flood graphic and not a counterclockwise spinny one, phew).  2008’s maverick was Ron Paul and not John McCain – can’t keep pulling out the same old shtick every four years.  I often wonder what happened to 2008’s ardent house flippers.  Hmmm, Brownie was a Bushie term of endearment in 2004 but “tsunami” wasn’t big until 2006?  I really like the evolution of key nouns and verbs across the decade.  Would you have done this graphic differently?

Thus, 2009 comes to an end.  The ox gives way to the tiger.  Here’s wishing all of you a safe rest of the year and a great 2010 filled with pleasant surprises and many new buzzwords to learn.  Peace.

If you don’t know already from some of my VizWorld posts, I’m a Flowing Data fangirl. Nathan Yau is the younger, hipper, nerdier Edward Tufte, and one who likes to share his sources and techniques. Understandably, Tufte has his trade secrets, but it was like pulling teeth to get him to share what tools and design methods he uses to make his graphics.  Something about Adobe Illustrator and a cadre of assistants is all I got.

Last night, I made a 2009 United States county-specific unemployment map using Flowing Data’s How to Make a US County Thematic Map Using Free Tools tutorial.  All you need is a Python installation, the BeautifulSoup XML parser, a good text editor and some patience to debug.  (Another reason I like Nathan: He codes in Python, the best, most intuitive programming language out there!)

These are the results, admittedly without a legend (bad Maitri!), which I will work on in Photoshop.  So you know what you’re looking at here, the lightest color is 0% unemployment and steps up from there in 2% increments, with the darkest color denoting 10+% unemployment.  This data was downloaded from the Bureau of Labor Statistics.

1. The Flowing Data original reproduced:

2. Diverging colors (blue=low; red=high)

3. Sequential colors (white=low; orange=high; black=+10%).  The darker the hues, the more trouble folks have telling them apart.  Black shows the worst hit spots and provides a backdrop with which to differentiate between the other colors

Check out the original Unemployment, 2004 To Present to see how bad things have become just in the last two years. This isn’t news, but just as well when you look at it in a county-by-county color graphic.  The nation is indeed bleeding.  Let’s make more casinos at home and start more land wars in Asia!

The events and artifacts of World War II fascinate me.  Not World War I, not Korea, not Vietnam, not even the war of the greatest import to my family, but World War II.  I still smile knowing we lived only a few blocks down the street from the D-Day World War II museum in New Orleans (incidentally, Jonah Langenbeck is the museum’s new Interactive Media Manager).  As my father-in-law, an American veteran of a foreign war, likes to say, “There is no such thing as a good war” and 1945 saw many sad, large, global messes in the name of victory, but I have nothing but awe for that era in world history.

The technology generated and used in World War II, that are still in use to this day, boggles the mind.  Radar, sonar, jet engines, rocket propulsion, nuclear fission and, most important to me, encryption and code-breaking.  So, imagine my surprise when D tapped me on the shoulder, pointed down the way to a collection of what looked like old, skinny typewriters and said, “Hey, you might want to take a look at those.”  In a large glass display case, on the starboard side of the U-505, sat naval Enigma machines recovered from the sub!  This one is probably a rare M3 with a ticker printer on top (the display’s captions aren’t too helpful).

Hope you like these pictures because it took me 20 minutes to get them (and the others in the set), I lost D And The Gang in the process and a search party was sent out to find me – sorry!  Crypotography is damned cool and how often does one get the opportunity to stare lovingly at a well-preserved Enigma machine?   Of course, my love for and comprehension of this topic  is … well, let’s just say I’m a dwarf standing on the shoulder of über-dorks.  I close this post by turning it over to Neal Stephenson, in his letter to mathematician Mike Anshel:

… As you know better than I, the Riemann Zeta function has been, and continues to be, of intense interest to mathematicians. During the 1930s, Alan Turing went so far as to build a mechanical device for calculating its values. This dovetails naturally with one of the chief themes of my novel [Cryptonomicon], which is the early history of the computer. So, in the book, I have invented two fictitious characters, Rudolf von Hacklheber and Lawrence Pritchard Waterhouse, both mathematicians who (so the story goes) befriend Turing at Princeton shortly before the outbreak of World War II.

A few years later, at the height of the war, von Hacklheber (who by this point has gone back to his homeland of Germany and has ended up working as a cryptographer for the Nazi regime) needs to invent a wholly original cryptosystem that has nothing in common with the Enigma, which he suspects has been been compromised. The system he comes up with, which is dubbed Arethusa, makes use of zeta functions. It is computationally intensive by the standards of the 1940’s, but this problem is ameliorated somewhat by the fact that, as a result of having helped Turing work on his zeta function computer at Princeton, von Hacklheber knows how to build a device that will automate many of the calculations.

As a supporter of Project Gutenberg’s eBook philosophy, I refuse to purchase a device that operates solely in proprietary file format and has hinky public domain vs. copyright and ownership issues associated with it. Lately, the PG-forum arguments for and against the Kindle have turned into ones of readability; subjective terms such as “comfortable” and “readable” are thrown about in place of that device’s accessibility and obsolescence. Personally, I have no problems reading the entire fraking Odyssey on my iPhone’s Stanza app after having downloaded it directly from Project Gutenberg. First of all, minimum eyestrain given that I practically live on my iPhone.  And other phones exist that offer comfort and reader resolution for long periods of time.  More importantly, it’s a special-super-secret-format-free, public-domain eBook which I can download again onto another phone once the iPhone is put out to pasture.  Ostensible readability is a sad metric with which to hold the Kindle up as the standard for eBook readers to come.

Back to the shady book ownership issues associated with the Kindle.  Do you know what I like about books more than readability?  The fact that they’re mine.  Once I purchase a book, I can do whatever I want with it: read it silently, read it out loud to myself or someone else, store it on my shelf for years, loan it to a friend, sell it or give it away to a library or school.  The same versatility applies to plain vanilla ASCII e-texts on computers and cellphones.  Can you do that with a Kindle eBook?  The answers range from definitely not to we don’t know, and this is why we cannot let Amazon’s Kindle or any other proprietary book reader establish the technological and legal standards for such devices.  Cory Doctorow writes in today’s BoingBoing (emphases mine):

Back in February, the Authors Guild, a lobby group representing less than 10,000 writers, argued that the Kindle’s ability to read text aloud infringed on copyright (it doesn’t — and even if it does, the infringement lies not in including the feature, but rather in using it; this is the same principle that makes the VCR legal). Amazon folded and agreed to revoke the feature.

Now comes some news about how they’re doing this, from the Knowledge Ecology International site:

“Beginning yesterday, Random House Publishers began to disable text-to-speech remotely. The TTS function has apparently been remotely disabled in over 40 works so far. Affected titles include works by Toni Morrison, Stephen King, and others. Other notable titles include Andrew Meachem’s American Lion, and five of the top ten Random House best-sellers in the Kindle store.”  I’ve been trying to get a statement from Amazon about this since February: how does disabling text-to-speech work? It appears that there’s a text-to-speech “flag” in the Kindle file-format that the Kindle looks for and responds to, disabling the feature if it’s set to 0 (a perl script called mobi2mobi can reset the bit to 1).  But what no one at Amazon will tell me is what other flags are lurking in the Kindle format: is there a “real only once” flag? A “no turning the pages backwards” flag?

This makes me a lot less inclined to purchase a Kindle than arguments about reader vs. phone readability and other straw-boogey-men raised by a few authors, their publishing houses and Amazon.  Think before you buy.  Your purchase has far-reaching media access and ownership implications.