Scripting News

People don't click links, that's why the 140-char limit will cripple Twitter

Tue, 08 Sep 2015 14:45:01 GMT

During the quiet holiday weekend a blog post from a writer named Startup L. Jackson got a lot of circulation. He says that Twitter's product is okay as-is, and people who say the 140-char limit must be eased are indulging in what he calls "magical thinking." As readers of this blog know, I am one of those people.

I don't say Twitter needs to ease the 140-char limit because it will grow the market, as Mr Jackson suggests, it might or it might not, but that isn't the reason. It's not a grow-the-market move, it's a defensive move.

I don't really want to repeat my last post on this topic, which ran just five days ago, but what the hell, one more time won't kill me.

Twitter has had real-time news more or less to itself since inception. Facebook was busy doing something else. Apple had the totally wrong idea of how news worked. Google had good products, Google News and Google Now, but they weren't doing exactly what Twitter does.
But things have changed. Facebook and Apple are actively pursuing news, and at least in Facebook's case, their product works better than Twitter's. Flipboard has an excellent product, and while they don't appear to be an immediate threat to Twitter, they could be acquired.
News products that are limited to 140 characters have to use pointers to guide the reader to the rest of the story.
Key point -- the new entrants don't have a 140-char limit.
If you think that clicking on a link to read a story is not a serious disadvantage, then go ahead and keep the 140-char limit. But Facebook claims to have done the research, and my anecdotal experience confirms this: people don't click links.
It's easy and non-disruptive for Twitter to ease the limit. The people who really love Twitter as-is will barely notice a difference. Except when they want to read more, they can just click a link, and the full story loads immediately, because the full article is already there, it's in the Twitter feed, just hidden at first. This is very simple, imho totally non-controversial stuff. Don't breeze by it, and think the limit is insignificant. It just cripples Twitter in relation to its new competition.
I've seen this happen so many times in tech. An early leader thinks there's something magic about their product, a newcomer enters and takes the market because they were wrong about the magic. Users almost always go for new power, esp when it comes to them as performance not complexity. That's all we're talking about here. News stories that load instantly as opposed to news stories that require for a new page to load.
It's pretty basic shit, Mr Jackson.

PS: To illustrate the point, I posted this story in full text on Facebook. Let's see which version gets more engagement, the one on Twitter, limited to 140, with a link, and the full text piece on FB.

Hunting for great MLB feeds

Tue, 08 Sep 2015 14:32:00 GMT

I love my NBA river, but it's kind of dormant now, since we're in the quiet period between professional basketball seasons. As the NBA is in hiatus, Major League Baseball is revving up for what's looking to be a really interesting postseason. I want to be prepared with a great river of news to keep me up on the latest gossip and speculation. I don't want to rely on the play-by-play announcers of the Mets games, which I am now watching as intently as I watch the NBA as it approached its post-season in April.

Anyway, if you're into baseball and RSS, you can help out, by joining me in the hunt for interesting baseball-oriented RSS feeds. On the first pass, I'm looking at regional news coverage, on a team-by-team basis. I already have good sources for the NY and Bay Area teams. Washington, Chicago, Pittsburgh, Houston, St Louis, Kansas City, Toronto -- feeds for the teams that appear to have a chance to being headed to the postseason.

Then I want to find interesting columnists.

Remember, they have to have a good RSS feed, or they can't play.

There's no give in that rule unfortunately.

One more thing..

Let's go Mets!

PS: The new river is starting to populate now. Here we go!

Why Twitter must blow past 140

Thu, 03 Sep 2015 21:15:13 GMT

Mathew Ingram has a post where he looks at the pros and cons of Twitter getting rid of the 140-character limit. I don't think there are any cons, they have to do it. Here's why.

Facebook is right. People don't click on links. It's not just true of mobile readers, it's true of all readers, everywhere, all the time. They. Don't. Click. Links. Memorize that.
Facebook wants to be in the news business. So does Apple. Probably a lot of other big tech companies. Will any of them have a 140-character limit? No need to answer that.
So there's Twitter with a package that can handle pictures, movies, vines, etc. But if you want to read a few paragraphs of text you have to click a link? If that limit lasts much longer it's an example of paralyzed management. Obviously there is no technical limit. If you can embed a video, you can include a few paragraphs.
The user experience would have to change a tiny bit. You'd see the first 200 characters or so, then a See More link, exactly like Facebook has. Or if they want to be a little more beautiful, they could put a triangular wedge there and allow it to expand and collapse with a nice animated effect. None of this is even slightly challenging to program.

Conclusion. They have to do it. What's amazing is that they've waited this long without doing it.

Baseball humor

Thu, 03 Sep 2015 16:07:56 GMT

I love going to baseball games because, in certain cases, it's okay to be incredibly rude to people, and it's funny.

For example, we were waiting on the train home from CitiField last night, reveling in a solid Mets win that included an inside-the-park home run, when a group of Phillies fans comes on.

I said something like "Sorry we humiliated you guys again." There is some honor in being a Phillies fan after all.

Then I noticed one of them was wearing a Yankees shirt. "What are you doing here?" A few seconds pass: "I guess they'll let anyone in these days."

PS: This started as a Facebook thread.

Update on Slack-like code

Thu, 03 Sep 2015 13:19:22 GMT

A brief note: The new version of nodeStorage, v0.78, is out.

This is the first version with Slack-like webhooks.

I have been able to use my existing Slack webhook code with this, both of which run unmodified with this server. So to that extent this is a clone, but I'm not claiming that at this time. They work like Slack, a subset of Slack's capabilities, and are not complete.

You can see the results of my testing in this liveblog post.

Progress on my Slack clone

Wed, 02 Sep 2015 14:40:29 GMT

I spent the last 1.5 weeks working on a very simple chat app that implements a subset of the Slack API.

It works!

I baked the server side of the code into a new version nodeStorage, which is a convenient place to put it because it already has full support for Twitter identity and of course storage, both of which come in handy for this app.

I'm not expecting to release the client soon. I may use it as a chat function for Scripting News, but first I want to let the whole thing settle in.

However, I will include the server-side for webhooks in a new release of nodeStorage. That to me is the really interesting part. Chat programs are commonplace, but such a nice simple API implemented in Node.js seems like it might have value to others. (Update: The new version of nodeStorage has been released.)

It's been a really fun project.

PS: I wrote about this project last month.

CitiBike goes north

Wed, 02 Sep 2015 12:37:25 GMT

Things are going to get interesting as CitiBike goes north up the west side of Manhattan.

The yellow markers in the map below are stations that haven't been installed yet. But some of the new ones have already been installed.

Places you'll be able to get to by bike: Lincoln Center, Zabar's, Barney Greengrass, the Museum of Natural History, the Metropolitan Museum, 72nd St stores, the Marina, a bunch of new places on the edges of the park. I only wish they put a station near the fountain in the middle of the park. It would probably have to be huge it would be so well-used. (And it would lead to lots of abuse by bikes on walking paths, which is probably why they didn't do it.)

Idea

If you have a restaurant or shop in Manhattan, west Brooklyn or Queens, pay for a CitiBike station outside your business, and let your customers know they can conveniently ride a bike to come eat or shop there. It'll be like having a subway stop in front of your store, soon enough. An investment for the future of the city.

Launching a Mac app from Node?

Wed, 02 Sep 2015 11:22:45 GMT

Back in 1999, I needed to keep Frontier running on a Mac server, so I wrote a little AppleScript that watched to see if the app was still running, and if not, launch it.

I need the same functionality today, but I'd rather use Node than AppleScript, if possible.

The question

Is there a way to make system calls to the Mac OS from Node that allow you to:

Tell if an app is still running.
Launch an app.

Any help much appreciated.

PS: Cross-posted on Facebook.

Update

Ted Howard posted a solution on the Frontier-User list.

It took a few tries but I got it working on my Mac.

Problem solved.

We're still in the Information Dark Ages

Tue, 01 Sep 2015 12:50:10 GMT

We're still not all the way there with social networking.

Facebook knows who I'm friends with, at least in a Facebook sense.

Now let's say I'm driving into San Francisco, going over the Golden Gate Bridge and wondering as I look at the skyline of the city, "Who do I know here?"

That's the question I had in 1979 when I was moving to the Bay Area from Wisconsin. Of course at the time there was no way to get that information, and I'm pretty sure it's still not possible today.

This was part of my definition of "information poverty."

I had another definition, and that part has been solved.

I was at my brother's house in Palo Alto in 1991. The Mets were playing the Dodgers in LA. Of course we weren't watching it or listening to it on the radio, there was no way to do that.

I asked a question -- is there a way we can find out the score in the Mets game right now?

We tried calling the stadium. Called the NY and LA news organizations. We even looked on Compuserve. No scores.

Of course today you can listen to the game anywhere in the US using the MLB app. And there are a million ways to get the current score. So by that measure of information poverty we're out of the dark ages.

PS: The Mets won again last night and the Nationals lost, moving the Mets ahead by 6.5 games in first place in the National League East. Lookin pretty good Mookie!

PPS: Colon pitched a great game through 8. But I have to say I do not like the uniforms they wore last night. I know support the troops, but please this is baseball, not combat.

PPPS: Another question I'd like to ask Facebook. Do I have any friends who live in DC who are Mets fans?

Twitter NT

Mon, 31 Aug 2015 11:54:09 GMT

On Saturday I wrote a piece about Windows NT, the operating system Microsoft created to run alongside the original Windows, as its eventual successor.

In a similar way, that's what Twitter needs. New technology, running off on the side, for pioneering users, the kind of people who were attracted to Twitter in the first place, the people who created the culture of Twitter.

Let's get the band back together, and this time, let's use what we learned the first time around to do it better.

New technology for Twitter

Here's what I would like to see in Twitter NT.

Easy programmability via a scripting language (JavaScript of course) that works directly with Twitter's API, in the browser. This is within reach now.
A more powerful data model. Let us attach any JSON structure to a tweet. There's plenty of prior art here going back to the original Mac toolbox, which allowed developers to add a refcon to a window. From that all kinds of amazing things blossomed.
Allow us to define in-browser callbacks that determine how a tweet is displayed, using the JSON data to either guide the rendering, or to provide the data being rendered. Instead of trying to force uniformity, let a thousand flowers bloom.
This new-technology-Twitter is walled-off from the celebrity version -- the one that entertainers, politicians, sports heroes and their fans use. Our tweets don't exist in their world, and vice versa. What we get is a solid back-end based on Twitter's cloud. And a license to explore and hopefully create the future.
Give it a few years to gestate, new ideas will flow from the place the hippies play to the news system of the future. This is where we've been aching to go with an Internet scale notification system.

In 2011, Twitter took a turn away from being a platform. This was, imho, the big mistake. Now the right thing to do is become the realtime platform that still does not exist on the Internet. Twitter is in the best position to provide it, for now. That opportunity won't exist forever.

Why you must watch BoJack Horseman

Sun, 30 Aug 2015 16:59:29 GMT

Note: There are a few nano-spoilers here, but you can safely read this and still have 100 percent of the fun watching the show, because nothing can begin to explain how spacy and intelligent it actually is.

This is more of a sales pitch than a disclosure.

Why you must watch BoJack Horseman

Did you like Breaking Bad? Well, Aaron Paul is one of the stars of this show. He's the guy who played Jesse Pinkman. He's even funnier in this show.
Did you like Fantastic Mr Fox? Well, this is a lot like that too, except there are are more humans, and instead of the star being a fox, he's a horse.
As in Fantastic Mr Fox, the animals know they're animals.
Do you like dogs? One of the main characters is a dog. He has a human wife.
How about JK Simmons, who played Dr Skoda on Law & Order? He's got a nice role, a turtle who is a Hollywoo mogul.
Speaking of Hollywoo, that's where this takes place. You'll be surprised to find out why.
Did you notice that the MSNBSea anchor sounds like like Keith Olbermann? Heh.
I'm going to compare this to Big Lebowski. I think in many ways it's as funny and daring as that great Coen Brothers comedy. There isn't a single character as great as The Dude, but there are so many of them, and it's about LA, and it's really funny.
J.D. Salinger is in the show, and he's funny!
Will Arnett plays BoJack.
Other actors, comedians: Stephen Colbert, Angelica Huston, Kristin Chenoweth, Stanley Tucci.

Netflix has another hit, but it's a sleeper.

Most people haven't heard of it. Yet.

PS: Cross-posted on Facebook.

Markus Persson and life after success

Sun, 30 Aug 2015 12:34:09 GMT

I read today's articles about Markus Persson with interest. I know the story, I experienced it myself a long time ago. Achieving all the success you wanted is not as simple as you might think. I've written about that before here on my blog, that's not the purpose of this piece.

The three tweets

Here are the three tweets everyone is quoting:

#1: "Found a great girl, but she's afraid of me and my lifestyle and went with a normal person instead."

#2: "I would Musk and try to save the world, but that just exposes me to the same type of assholes that made me sell minecraft again."

#3: "People who made sudden success are telling me this is normal and will pass. That's good to know! I guess I'll take a shower then!"

It's #2 that's the most disturbing to me.

Why couldn't he have continued to work on Minecraft?

I ask this question because I did the same thing with blogging and RSS. I walked away from it because defending against personal attacks got to be all I was doing. The people who were trying to trash my rep were experts. I had forgotten how bad it was, until (also yesterday) I read my blog archive from late August 2005. I had pointed to it because of the ten-year Katrina anniversary. But also intermixed in there were my responses to all the trashing. No, there was no justification for it, other than people didn't want me to be able to do what I was doing, I guess. Or they were jealous. Or who the fuck knows. All I know for sure is that I wanted to work with other people, and there were people working in the shadows making sure that no one would work with me.

I think the mistake successful tech people make that people in other arts don't is we tend to believe in the communication tools, and we want to use them the same we did before we became targets. But as your influence, fame and fortune grow, so does your follower count, and so does the hate.

A guy who was able to create something as wonderful as Minecraft should, if he wants to, be creating the next thing after Minecraft! Did the Beatles stop recording after Sgt Pepper? We need to create safe contexts for our superstars to keep doing what they love to do.

Windows NT

Sat, 29 Aug 2015 15:47:53 GMT

Flashback to 1995

It's roughly the 20th anniversary of Windows 95.

I was there, in Redmond when it rolled out. With Jay Leno and Bill Gates. I got a private demo of Blackbird, which was supposed to be Visual Basic for the Internet.

Quite a day. It was the one time that Microsoft got an Apple-style euphoria for a product rollout. Long lines outside retail stores. They all wanted to buy a box with a CD inside. Back then you didn't download software from the net.

Windows 95 was the second to last release of Windows "old" technology, based on MS-DOS, if I recall correctly. Off on the side they had a project called NT. It was the future. On that OS, apps were protected from each other. On Windows OT, if an app went crazy it could kill the OS, and all the apps, and lock up the keyboard and screen. You had to press Ctrl-Alt-Delete or pull the power to reboot to continue using the computer.

OT was splashy, but it crashed. Jay Leno probably used OT. 20 years ago it was everyone was so excited about.

New technology for Windows

NT was cool and wonderful. I was a Mac user, but in a couple of years I would be an NT user. I never used Windows 95 or its descendants. I had used earlier versions of the OS, and wrote software that had to defend against hostile competitors that took advantage of the free-for-all OT approach.

NT was good. It was built on the tried-and-true principles of minicomputer and mainframe OSes, not from the wonderful free-for-all days of the early PC (it really was cool to have the whole machine at your disposal). Neither was bad or good, but the base to build on was the one where software couldn't go to war with other software, intentionally or unintentionally.

Why blogs are hurting in a nutshell

Tue, 25 Aug 2015 17:12:10 GMT

Another nutshell post!

Twitter says posts must not have titles and can't be longer than 140 characters. Posts cannot contain HTML markup.
Google Reader said posts must have titles, and are assumed to be essay length. Posts may have HTML markup.
Facebook says posts may not have titles or markup but can be as long as you like.
No discourse on Google Reader, grunts and snorts on Twitter, good engagement on Facebook.
None of these are great for receiving all of what we were posting to our blogs before they came about. The APIs are inconsistent, but at least they have them.
Along comes Medium, which could be great. They handle markup, titles, and lately no titles required. Any length. Great! But no API. Oh geez.
Tumblr and WordPress do a pretty good job of holding onto the energy of blogs, all things considered.
Just imagine if one of Twitter and Facebook had tried to really harmonize with blogs, how much smoother everything would be. We could have archives and post to our friends, who could read our gems in place without clicking on links, which they've proved over and over in so many ways they don't like to do. (How many times do you get comments on Twitter and Facebook that react to the title, ignoring the content probably because they didn't click.)
There seems to be some hope Facebook might put an API on their upcoming Medium-like service, or perhaps this will inspire Medium to put an API on theirs. Or Twitter could ease up on the 140-char limit, allow markup. Any number of things could put blogging back in business.
It would be nice to have a friend among the Silicon Valley tech elite.

Twitter in a nutshell

Tue, 25 Aug 2015 16:05:02 GMT

O'Reilly has a series of books entitled X in a Nutshell.

They're called that because presumably nut shells are small things. So if you want to understand something in a nut shell, that means you're getting a distillation, a summary, a conclusion without the reasoning.

We used to call them Busy Developer Guides. Because you're busy and you just want to know how to do something.

With that windy preamble, here's the story of Twitter, in a nut shell.

The story

Twitter is the news hub for news makers and news vendors.
Kind of like GitHub for news.
Now Facebook wants to be that, and is moving aggressively.

The future

As before, in a nutshell.

Twitter's challenge is technology evolution, not Wall St.
Once users are excited about Twitter again, investors will forget metrics.
Summary: Make the news work better in Twitter, again and again.

Another requirement for future-safety

Mon, 24 Aug 2015 14:15:05 GMT

How would you feel about an email service that didn't allow you to forward all your email to another account?

Yet we post our writing and photos to sites that don't provide the ability to redirect to the new location of our work if we think it's time to move.

So I'd add another requirement. It's an either/or.

Either you let me point my own domain to my content on your service, or
You provide the ability to redirect from my section of your site to whatever site I choose, for the indefinite future.

I suspect a vendor who wanted future-safe certification would likely go for #1, it's a lot cheaper. The "indefinite future" part in #2 means they'd probably have to buy insurance of some kind (a product that doesn't yet exist, but should).

PS: See the Aug 17 piece about criteria for future-safety. There's a lot more to this.

Mac OS is spyware too

Mon, 24 Aug 2015 13:57:08 GMT

All of a sudden my Mac is telling me whose birthday is tomorrow. People I don't even know that well. How did that happen.

I don't like my computer randomly and unpredictably getting all "social" on me. It's a tool.

Try to imagine a carpenter's hammer starting to nag about an upcoming bar mitzvah.

A baseball player's bat starting to warn you about overdue bills.

Who asked for this shit!

PS: Here's the Facebook thread that inspired this post. Interesting comments.

The XML-RPC site works again

Mon, 24 Aug 2015 03:27:55 GMT

There was a lot of breakage in the old XML-RPC site dating back to 1998. I finally have all the tools I need, I think, to get it working again.

Nice to be able to get this stuff working again. With all that I write about future-safe archives, it's kind of funny that some of the real history I am supposed to be managing is in such poor shape.

At least this little part is in better shape now.

PS: There are still a lot of broken links. Which is what you would expect from a site that's 17 years old.

PPS: Here's the first link on archive.org that has useful information.

Do people still use libraries?

Sat, 22 Aug 2015 02:25:08 GMT

John Palfrey, a former colleague at Berkman Center, has a new book about the importance of libraries in the age of Google. Yesterday it got a short writeup in the NY Times. I haven't read the book yet but I plan to.

When I was a kid, I spent a lot of time in the Queens public libraries. Most often in a local branch, but sometimes when I needed access to more information, particularly microfilm of old newspapers, I would go to the central library in Jamaica. It was a long bus trip, so I must have had good reasons to go.

I also went to the library as a sanctuary, a way to get away from difficult situations at home. I suspect a lot of kids used libraries this way. A place where the adults were friendly, where it was their job to help you, and they really seemed to enjoy it. Librarians helped cultivate my interest in knowledge, something I am very appreciative for. The librarians also took us seriously, something a lot of adults don't do. We fail to understand that kids are real people. We get confused by their cute faces and small bodies, their childish enthusiasms. The librarians' job was to encourage the intellectual seeker and explorer in every child.

In college I spent a lot of time at the main library at my school, as a quiet place to read on campus. I didn't really have a good place to study in the various apartments I lived in. Maybe that's still true for college students?

These days I don't spend much time in libraries. Sometimes I go there to write, because it's a place that gives me ideas, but all the information I would get from a library I get on the net. Of course it's a lot faster and easier. What's missing on the net is serious collaboration. There isn't a place on the net, that I know of, where people are serious, where they try to do their best thinking, and come with open minds prepared to learn from others. Most of what you see are poor attempts at humor, outrageous political opinions. So many examples. This morning a friend on Facebook posted a link to a serious story. Every comment under it was a one-line "joke" none of which were even slightly funny, imho. This is the norm. The net is where the idea of Trump as president originated, where it thrives. I want to go some place where the idea of solving problems collaboratively is on-topic, and snark is frowned on. The net has great reference works, but it isn't a great environment for serious thought and discourse. As one of the early Internet "utopians," I never imagined we'd get to this place.

Perhaps this is the future role of the library. It's a physical place. In the past it was also the place to seek information, when information had a physical embodiment. But now that information has been disconnected from books and microfilm and stored digitally on disks and SSDs in the cloud, what's left of the library is the philosophy and people. They are still open meeting places for people.

I have to make some trips to libraries to see what's going on there in the evenings and weekends. Who's there and what are they doing? What do librarians think the role of libraries are? For me, once, they were incredibly valuable places. Do people still use libraries?

Ideas

Lots of interesting ideas in the comments on this post!

One thought came to me, how about putting a podcast studio in every library?

People could come to the library to interview family members, to record their stories, to be saved by the library.

Which led to the idea of a network of libraries hosting blogs for everyone, using rules that are fair for people, not designed to create profits for tech investors. The stories have so much more value than the few pennies they make for tech billionaires. It would be great to have a public institution that helped people tell their stories, without any conflicting interests.

Maybe this is something Bill & Melinda Gates would fund, following the example of a great American philanthropist of the early 20th century. It seems it's time for an update to the concept of the library.

I would love an open source Slack clone

Thu, 20 Aug 2015 15:53:57 GMT

Last Sunday I saw a tweet from TechCrunch founder Mike Arrington that said he missed the old Silicon Valley and might try to do something to bring it back. I was enthusiastic, and suggested what the tech industry needed was a new open platform to grow on. I was thinking of the IBM PC in the 80s and the web in the 90s. What Twitter might have been in the following decade, had they not punted.

Mike responded that he can't afford to do it, to which I asked if he was an investor in Slack, one of the famous unicorns of the new tech industry. A startup with a market cap in excess of $1 billion. It is both an open platform and a financial juggernaut.

Let's think about Slack

That got me thinking. What would it take to fully develop Slack as an open platform, even beyond where it is now. The answer came to me right away. If Slack is the IBM PC, what we need is the Compaq. Or if Slack is Netscape, we need MSIE (the early versions of course, not the malware-infested wasteland that MSIE turned into).

It must be a clone

Back in the heyday of PCs, the first round of PC competitors were near-clones, they could run PC software that was modified to work on their systems. The differences weren't huge, but they proved to matter. The near-clones eventually fell by the wayside, because if they didn't run PC software out of the box, users didn't want them.

The Compaq PC ran most IBM-compatible software out of the box, unmodified. I was a PC software developer at the time, we totally appreciated not having to create a new version for each PC competitor that came along. Compaq grew like a unicorn, and the IBM PC kept growing along with it.

I think Slack is big enough and important enough that it could serve as a foundation for a great new open ecosystem. A good Slack clone would have to work with the existing base of webhooks, unmodified. Exactly as they are. "Out of the box," as we used to say.

I don't think this would hurt Slack-the-Company at all. They are clever and moving quickly, and most important they understand and love their users. That's what it takes to maintain leadership of a market. The users have to think of you as the "official" platform, and the others as clones. If they hold to their principles, and I don't see why they shouldn't, the users won't be fooled.

Personally I'm not particularly interested in cloning the user experience of Slack, however I am interested in being able to run their webhooks in other environments.

Exactly what is needed

Run incoming and outgoing Slack-compatible webhooks unmodified.
It must be open source, MIT license or equivalent.
Written in JavaScript.
No frameworks, no dependencies.
Runs in Node.js on the server, in browser-standard JavaScript on the desktop.
The layer that runs the webhooks must be a cleanly factored separate module, not integrated with the UI, so it can easily be incorporated in other kinds of software.

I can't think of anything else at this time, can you?

Update

Stewart Butterfield, co-founder and CEO of Slack replied: "I would like this! We'd need to clean up our APIs a bit (working on it) and add a few simple capabilities. More the merrier!"

That's really cool. I was pretty sure he'd go that way.

Update August 29

I've got less than 1/2 of the webhook API implemented.

There's a lot of text processing that goes on when Slack receives a webhook call. I have a little of that implemented. But I do have an app that receives incoming webhook calls, and does the correct thing with them, distributing the data to all the clients that are hooked into the server.

I have to say the API is as clean and sensible on the server side as I thought it would be from implementing a client. Slack has a very practical engineering culture. I'm totally enjoying the work.

Car insurance price based on credit score, shopping habits

Wed, 19 Aug 2015 12:30:41 GMT

Consumer Reports says that car insurance pricing doesn't work the way it used to. A couple of big differences.

Your credit score is a bigger determinant of insurance price than your driving record. A person with a high credit score and a DUI conviction can get a lower insurance rate than a driver with a perfect record and a low credit score.
Your sensitivity to price also determines your rate. They get this information presumably from credit card companies, supermarkets, department stores, perhaps Amazon. If you tend to buy more expensive things and don't do a lot of price comparisons (something Amazon would know, for example) they give you a higher price. That's why if you call them and ask for a lower price, often you'll get it. This practice is illegal in a few states, it should be illegal everywhere.

I found #2 shocking, and wanted to share it because I personally didn't care if companies knew what brand of bottled water or canned soup I buy, but now I can see why it costs me money that they do know. If you're a carefree shopper, it costs you more ways than you might think.

Bezos on kindness and the NYT willfully tells an incomplete story

Wed, 19 Aug 2015 12:30:33 GMT

The New York Times ran a remarkable piece about what it's like to work at Amazon HQ in Seattle. I read the story from top to bottom, and was, like a lot of people, fairly disgusted by the way they treat people who work there.

But there was one story that stood out, near the top of the piece, that seemed out of place. Here's what the Times wrote:

He wanted his grandmother to stop smoking, he recalled in a 2010 graduation speech at Princeton. He didn’t beg or appeal to sentiment. He just did the math, calculating that every puff cost her a few minutes. “You’ve taken nine years off your life!” he told her. She burst into tears.

I'm always leery of such obvious appeal to emotion. He made his grandmother cry. He must be a bad person. But he was just a kid. What's significant is not what the 10-year-old Bezos said and did, he wasn't running Amazon, rather what the adult Bezos said, which the Times left out of the story.

I sat in the backseat and did not know what to do. While my grandmother sat crying, my grandfather, who had been driving in silence, pulled over onto the shoulder of the highway. He got out of the car and came around and opened my door and waited for me to follow. Was I in trouble? My grandfather was a highly intelligent, quiet man. He had never said a harsh word to me, and maybe this was to be the first time? Or maybe he would ask that I get back in the car and apologize to my grandmother. I had no experience in this realm with my grandparents and no way to gauge what the consequences might be. We stopped beside the trailer. My grandfather looked at me, and after a bit of silence, he gently and calmly said, "Jeff, one day you'll understand that it's harder to be kind than clever."

That bit of wisdom, which we can spend much of our lives learning, would have offered an interesting counterpoint to the thesis of the Times piece. Why did they leave it out? As a lifetime NYT reader, this really requires an explanation, otherwise you have to assume the reporting in the rest of the piece, and in other NYT stories, was just as deceptive and partial.

The NYT public editor, Margaret Sullivan, wrote about the Amazon story, and provided the full quote, yet she says the Times didn't get any of the facts wrong. "No serious questions have arisen about the hard facts." Might be true if you overlook this clear omission. Maybe not a "hard" fact, if so imho that's an irrelevant hair-split.

The Times could be so much more than it is, but at times like this it appears to be even less than we thought it was. An omission like this invalidates the rest of the piece. If they can be caught being so manipulative of readers so easily, what about the harder parts, where they quote anonymous sources. How can we know if they omitted important, relevant parts of their stories? We can't, so we have to assume they did.

As always, when they are so manipulative they lose credibility with readers. And this really is their only asset.

Future-safety notes

Tue, 18 Aug 2015 14:49:50 GMT

A few random items for the future-safe web file.

A commenter on yesterday's post asked about my personal profile page from 1998, which was linked to from the RSS 2.0 spec, which I pointed to as an example of something that seems relatively well set up for longevity. The link was broken. I investigated, and found that it had been broken since I moved from Apache to Amazon S3, a while back. It was a perfect example of how easy it is to lose large amounts of web content. On Amazon S3 I set the default filename to index.html, and on Apache I had set it as default.html. This was almost certainly a mistake. I fixed this link by creating a copy of the home page at index.html and now it works.
Case-sensitive servers are bad for future-safety. I wish Amazon S3 had the option to tell the web server to be unicase. I ported scripting.com from a server that didn't care about case, Apache on Windows. A lot of the broken links in my old content, migrated to S3, would be fixed if I could set such an option true. It's a lot more difficult to fix with a patch, because folder names need to be case-insensitive too. Discussed in this Facebook thread.
Facebook "notes" could be a boon for future-safety or a new disaster. Yesterday I got lucky and spotted a new Facebook style of notes page, one that looks a bit like the pages produced by Medium. It got a ton of coverage in tech pubs, who saw it as Facebook wanting to attract bloggers. This could be a great thing, or a terrible thing, depending on how good the API is. If I can produce a version of MyWord that works with both Facebook notes and pages on the open web, then we'd have a fairly future-safe system, and one that others can build on. The best of both worlds. But if there is no API, we get another Medium, a nice-looking landfill for ideas, although we believe more in the longevity of Facebook than we do of Medium.

The "President Reagan" Show v2.0

Tue, 18 Aug 2015 13:28:01 GMT

In this clip, Brian Stelter. who has a weekly show on CNN about the news industry, warns reporters not to pay too much attention to the polls in the Presidential "horse race." He's doing this the way a blogger would. Makes me really respect him. Pop out of the bubble says Stelter and look at past experience as a guide to how real any of this is.

There's always a sports analogy

I'm a Mets fan, all the way back to the beginning, but I wasn't paying attention this year, too focused on other things, but then there was this story about Wilmer Flores, a Mets infielder. He thought he was being traded. He was signed by the Mets when he was 16, so I guess he grew up on the team. He was crying. On the field, on camera. The next day he hit a walk-off home run. The whole team came out to greet him at home plate. It was a very emotional moment. And that was the beginning of a huge turnaround for the Mets. They're in first place now. (But being a true Mets fan I'm fairly sure it won't last.)

We Facebook-like Trump

Same with politics. Human emotions go in waves. We "like" Trump now because his story is interesting. We want to see how it comes out. But as Stelter points out, don't mistake Trump's rise as necessarily significant for the election that's still 447 days away.

I put "like" in quotes above because the way we're interested in Trump is the way we click "like" in Facebook. Sometimes you click something you don't like. Just to bump it up and tell the algorithm you want to hear more about this. TV has similar signals. They themselves are interested to hear what Trump will say next, so they figure we are too (we are). They probably run focus groups to stay close to where audience interest is.

Emotions flow in waves

The last two Presidential campaigns in the US were after the advent of social media, so human waves happened then more like they happen on the net, which is different from how it worked on 20th century TV. It's more wave-oriented, shorter attention-span. But a good long-running meme like Trump obeys new and different rules. The story probably won't flow like Giuliani in 2008, or Bachmann et al in 2012, because Trump has been laying the groundwork longer, he's more interesting, and knows how to keep the interest going. And the net keeps evolving. It changed a lot between 2008 and 2012, and it's evolved from that today.

But Stelter's point is still interesting, and optimistic. It would be good if we could avoid "President Trump." But I was around in 1980 and had much the same feeling about the concept of "President Reagan" and uhh, well, let's hope we've learned.

The President Reagan Show starring President Reagan

Today Reagan is offered as the Republican paragon of American leadership, but he was actually a fair actor who played the role of President in a TV show. He had nice hair. If only they could find an actor like Reagan. But most people my age, before he was elected, thought Reagan would be a disaster. We thought he was as electable as Trump is today (i.e. not). Even more support for Stelter's thesis, not to be too swayed by what's happening in any moment, but also not to look to the past as a guide to the future, too much.

Program notes

Note to CNN: Why not allow video embedding as YouTube does. I bet you'd get more circulation. Remember, blogging is on its way back. Time to hone your content to take best advantage of that.

Another note: Every product should have an easy-to-find page with logos and product shots designed for including in blog posts. They should have transparent backgrounds, be approx 145 pixels wide, and roughly square. A variety of sizes actually would be useful. For candidates and media personalities, nice web-friendly head shots. I asked for this years ago, it hasn't yet materialized. I also asked that advertisers provide their commercials online so we can point to them, and while that isn't systematic yet, you pretty much can find what you're looking for.

Criteria for future-safety

Mon, 17 Aug 2015 11:21:49 GMT

I've written about a future-safe web many times -- the idea that what we're creating on the web should persist. Will the ideas we publish be there years from now, so others can know who we were, what we did and what we thought?

The concern is that the record we're creating is fragile and ephemeral, so that to historians of the future, the period of innovation where we moved our intellectual presence from physical to electronic media will be a blank spot, with almost none of it persisting.

If, for example, this website were to persist, you would be able to read these words, at their permanent address, many years into the future.

Criteria?

I made a list of some of scenarios that illustrate what I mean by future-safety.

Some sites say you can download a full copy of all your work, but if the format is proprietary that doesn't count for much, and if they make it hard to do, or slow, that takes points off as well. There has to be some easy way to do something with the downloaded content.
The highest-rated system would be one that's hosted in static HTML on the server of a long-lived institution. For example, I think the RSS 2.0 spec is well-situated for longevity. It's hosted on a static server at law.harvard.edu, along with other static content for the law school. Harvard has been around since 1636, that would seem to bode well for it being around in 2115, one would hope, and perhaps 2215? The Library of Congress, established in 1800, would be an excellent place to put long-lived public hosting.
Dynamic content on my Windows 2003 server running on Rackspace would get a low score for longevity.
Something you publish on Medium would get a low score, since the content is part of the business model of a revenue-free startup that's raised a huge amount of money. Chances that the writing survives long-term, relatively low. And Medium is especially dangerous because people are storing historically significant writing on their servers, with no provisions for longevity.
If a service such as Medium offered a chance to mirror content on another site, that would dramatically improve the rating. If it were automatic, default-on, and the mirroring site was a static site of a long-lived institution, it would get the same rating as the long-lived site.
Another way to achieve longevity would be to add an API that allowed it to be part of an openly implemented web content management system. That way other developers could implement mirroring from the private site to a public, static, long-lived one.
A fantastic case-study is Sourceforge, last generation's GitHub, that's now putting malware in the archives (according to reports). When authors remove repos from their server, they put them back. Open source licenses make that possible.
Services like wordpress.com and tumblr.com would get relatively good ratings because they have extensive APIs allowing them to be part of open content systems. APIs can be used to correct a lot of sins.
No one today would get a perfect score because there's no way to purchase (as far as I know) a service agreement for the indefinite future. That would be a truly future-safe service, if we believe that the vendor is long-lived. (That's a key factor, this is not a service that can, imho, be run by a startup.) The agreement has to include renewing the domain name the content is hosted on.
One more thing, a lot of people say they don't care about future-safety, but there's no reason not to care. Essays are tiny capsules of knowledge compared to video and audio, it is very inexpensive to store writing in ways that it can survive long into the future. If users want this, we will have it. If we had a rating for every service, you could pick and choose based on this feature as well as others that matter: readability, distribution, ease of editing.

Update

On August 24 I posted another requirement for future-safety.