Vision for the theme directory: Part 1

I’ve now been involved in some form or another with the theme review team at WordPress.org since late 2010. Some of that has been as an author, reviewer, and administrator. I’ve been a part of every aspect of the process. I’ve been frustrated as a theme author. I’ve been worried by code quality as a reviewer. I’ve spent numerous hours thinking about solutions as an admin.

I joined the team because I wanted to improve the process so that both other theme authors and I could get our themes out there without a lot of fuss.

What I found was that over the years, there are really a lot of people doing some sneaky things. There are a lot of security issues. And, many authors include code or assets that are not compatible with the GPL.

When you see that, you get in the mindset of assuming everything’s bad. Starting from that assumption is not always a good thing. Guilty until proven innocent. My vision of the theme directory is to come from a more optimistic viewpoint.

The ideas presented in this post are by no means all my own. They’ve been shaped and reshaped by users, theme authors, and reviewers over the years.

“That” feeling

There’s this slightly euphoric feeling when you first finish a theme. You’ve done the work and finally decided it’s time to share your work with the world. If you’re a theme or plugin author, you know what I’m talking about.

There’s a ticking clock on this feeling. If you’re not able to share your project within that window, you lose that small bit of joy of sharing your theme.

The theme review process has been killing that feeling for theme authors. When you wait for months for your theme to go live, all you’re left with is a meh feeling. There’s no more excitement about your theme. You’ve most likely already moved on to another project.

Meh

For first-time theme authors, this is even worse. Some of us have been there and done that, but for the new theme author, the process can be a huge stumbling block to someone who might become the Picasso of the theme world one day.

The theme review process

Over the past couple of years, it’s been painfully obvious that the manual review system (in its current state) simply cannot keep up with the high volume of themes.

As a reviewer, I know that many theme authors are not educated enough in a few areas:

  • Basic security, particularly with output escaping.
  • Licensing and understanding what’s compatible with the GPL.
  • Core WP functions or features that already exist.

The question has been about finding a balance. Security and licensing are definite blockers. I think everyone can agree that themes should be secure and that anything promoted on WordPress.org needs to be GPL or compatible. But, the other stuff is not so clear cut.

Should theme reviews require that theme authors use the core-bundled jQuery script? I know many plugin authors who certainly think so.

Should theme reviews require that the core WP the_content() is used in favor of some custom function to show post content? Again, it’d break numerous plugins if it didn’t.

Should reviews check that theme options actually work? I’m sure there are users who appreciate it.

That’s where a whole boatload of time is going. A lot of people are in favor of scaling back theme reviews to security and licensing. Then, allow users to up-/down-vote themes using the .ORG ratings system. It makes sense in many ways.

Long term, it may be the only way to make the process work without a fulltime, paid staff handling theme reviews.

I’m absolutely certain that this will break more plugins. I’m absolutely certain that it’ll increase .ORG support forum posts. But, if users truly use the ratings system, it could correct itself to some degree.

We can also continue building on the Theme Check plugin to block unwanted code. If we open the floodgates, it could have the benefit of us building the best code sniffs in existence.

Right now, it takes at least an hour to do a full check of some of the more basic themes. It can take several hours to check a poorly-coded theme. That’s not a good use of volunteer time. That time would be better spent improving the theme directory in other concrete ways.

Where do we go from here?

As a user, I appreciate that themes are checked for my benefit. However, I know how to give 1-star ratings.

As a theme author, I want to open things up. Check only security and licensing.

As a plugin author, I weep at the thought of all the themes that will break my plugins while relishing in the freedom that some theme authors have never known.

As a reviewer, I’m tired. We have too many reviewers who get burnt out trying to keep up with the load.

My official recommendation is for the WordPress.org theme review team is the following:

  • Only manually check security, licensing, and any other egregious issues.
  • Keep building on the guidelines because theme authors really need those.
  • Continue to iterate on the new Theme Check to catch the stuff we don’t want.
  • Give users the freedom to rate themes as good or bad.

Relying on Theme Check

We’re programmers. We write code.

It doesn’t make sense that we’re manually checking for code issues when there are tools available to automatically handle most of the load.

In order for us to make sure that themes are secure, it means making the “escaping” check an error-level notice. Basically, this means that anything Theme Check notes as an escaping issue will automatically block a theme from being submitted.

Previously, I was in favor of leaving this as a warning and manually verifying something as an issue. That puts work on the reviewer. Making it an error-level notice will put the responsibility back with the theme author, where it rightly belongs.

Even if we have false-positives, it means that theme authors will be forced to rethink how they write their code, which is mostly a good thing.

Building the guidelines

We should not do away with the guidelines. These should be the official recommendations of the team. They should also help us mold Theme Check so that it continues improving to catch the bad.

The guidelines should also be the basis for the entire handbook. The handbook should be filled with code examples of best practices.

Primarily, security and licensing guidelines should be front and center. Reviewers should manually check those. The rest are there for theme authors to follow on their own and for the team to build into Theme Check.

Side note: There’s also a scenario where theme authors might request a full “guideline review” to get a seal of approval or some sort of badge for their themes if we wanted to go that route.

Featured themes

Currently, featured themes on the WordPress theme directory are randomly chosen. This was done to make things fair. This is in large part due to some gaming of the system in the past.

I’m in favor of the admins and/or moderators of the theme directory making the decision about what themes appear in the featured themes list. This would be a curated list of themes that we think represent the best of the WordPress theme world.

No, this is not fair to everyone. I’m not arguing for fairness. I’m arguing for some of the brightest developers and designers in the WP community to hand-pick awesome themes that they notice coming through the system.

Right now, too many great themes get lost because they can’t get noticed. A big part of the team’s job should be making sure users can find these diamonds in the rough.

Trusting users to rate themes

Most of this comes down to users making liberal use of the ratings system. When they come across a theme breaking their plugins, they need to give bad ratings. When they come across a theme that works perfectly, they need to give a good rating.

The theme review team has spent much of its time protecting users from a lot of bad code and practices. But, we have to put a little responsibility into users’ hands too. In my experience, users are pretty quick to let you know about problems.

So, let’s give them the chance to do it.

If things will be bad, why do it?

I’ve outright stated that if the review team only checks for security and licensing issues, themes will absolutely break plugins. There’ll be many things that simply don’t work as they should.

Why in the world would I want that?

Because for every X number of bad themes, we’ll have a truly amazing theme that we might not get otherwise. If that’s 1 in every 100, it’s worth it to me.

I want us to open the process up and see what people can come up with.

Steps to achieve the vision

First, we need a solid Theme Check plugin that can find more security issues. That’s primarily a matter of getting the new sniffs finished.

Second, we need to complete the guidelines rewrite project. Even though reviewers wouldn’t manually check all of these, the guidelines should still be a guide to coding quality themes.

Then, it’s just a matter of pulling the trigger. Pick a date and make the switch to only reviewing the absolute essentials.

I’m ready. I know most theme authors would jump for joy at such a drastic change. Now, it’s just a matter of whether the team goes for it or can build upon this vision.