Drupal 7 admin, some thoughts

Introduction

I keep labelling posts with pt1 and I do intend to re-visit them, however getting tired with that so this is labelled with 'some thoughts', I can always get back with some more thoughts at some stage. Basically for my own consumption, I want to come back later and see where the land lays regarding admin in Drupal 7 and compare to Drupal 8 as appropriate.

Admin interfaces and user experiences are improving, one or two legacy Microsoft based cms interfaces I have had the pleasure to play on were truly appalling things that only worked properly in IE etc. etc. and just made you feel like not wanting to add/edit content rather than making it easy for you to do so.

In light of the above I can see where Drupal administration can seem like a pleasant breath of fresh air, however compared to where it could be, if marking a report card for a student I think Drupal would get a "could do better". When I come back with some more thoughts then I will make a few comparisons but just for now, Wordpress is better, feels much more intuitive. If anybody who has used both wants to argue the point I would be happy for your comments. Note I am not saying Wordpress is better and I recognise that Drupal has a bigger scope but the admin in Wordpress wins out in my opinion (and many others).

Two well known tweets sum up the perceptions around Drupal vs Wordpress:
Dries: RT @chx1975: WordPress is now approximately where Drupal was around Drupal 5 w/ content types. See you in 2015.
and
Dries: Agreed, and fair enough. RT @newoceans_en: @Dries Drupal 7 will hopefully be where WordPress was around 5 years ago regarding UX.

Handing over a Drupal application to a client who has experience of Wordpress can be a little painful, to take a quote from Katherine Senzee when being interviewed about the Overlay (mentioned and linked to below).

I chose it (Drupal) because it was developer friendly, now it is kind of funny that I am spending my time making it end-user friendly.

Some embarrassments

Media assets

Core Drupal leaves something to be desired in the media department, this is to some extent addressed by the Media module. A contributed module, that adds a lot of functionality, however a number of issues will be addressed in version 2 of the module which is still unstable (unfortunately). Some enhancements were made via the media browser plus module but this project doesn't appear to be keeping up with or working well with the media module.

Perhaps things will look a little better on this front in a month or two.

Taxonomies/Menus

These are all by default rather awkward to administer and apply when the taxonomies or menus are large and the widget options for editing don't provide you with something that can easily be handed over to a client.

The overlay

Maybe some people like the overlay feature, but generally it seems to cause occasional errors, hard to work with clients that have old crufty machines or browsers etc. and sometimes errors depending on what other modules you have installed (in my experience). You can hear a bit of a discussion about this here in a Drupal voices podcast..

Maybe I need to spend more time with it, but the overlay doesn't seem to offer me anything that aids in quickly helping me train clients to administer their Drupal site, by default I find myself disabling the overlay module.

Some useful helpers

Dashboard and Shortcuts

The customizable dashboard and shortcuts are a terrific boost to quickly making a site administration better to hand over to clients. Shortcuts to things that they have permissions to use appear nicely in the gray bar (assuming you are using the Seven administration) theme. You can add blocks to the dashboard and make it a the main link they can go to, to re-orientate themselves.

Views

Maybe after one of my previous posts this may seem surprising but views really come into their own in admin, you can provide a tool for the end-user to query their data, very easily with views. Simply creating some of these views and providing a block with links to them in a block on the admin dashboard can go a long way towards making a better admin experience for end-users.

Drupal framework learning curve pt1

Introduction

Drupal is a development framework for people who know PHP and know Drupal, one step more than a pure development framework

Another start to a multi-part posting, this time working in on something that took a little longer to occur to me than perhaps it should. Drupal has a big learning curve for anybody who needs to do custom development. Drupal is often described as a cms that is moving (or already moved) to being a development framework. To use Drupal as framework however requires a good knowledge of Drupal (duh obviously, but bear with me).

Typical frameworks

Development frameworks are typically built on top of a programming language. So for example you may be a PHP developer who picks up a new framework built on PHP or a Ruby programmer who picks up a framework built on Ruby.

Development frameworks are typically built using well known design patterns. The MVC pattern being a prime example (often mistaken by recruitment consultants as an actual framework in it's own right). To this extent experience with one or more frameworks can boost the uptake of a new framework. For a developer or team that has experience with relevant technologies, the programming language and some of the patterns used, new frameworks can be picked up quite quickly.

Drupal as a framework

To use Drupal as a framework you need to know Drupal, you need to know PHP, you need to know about a lot of contributed modules that extend Drupal's core technology, you need to know about the changes from earlier versions (to help make sense of some of the documention that is there) and you need to have spent time learning where to find help and further information. Drupal has nothing to compare with the documentation of many other frameworks.

Granted that pure frameworks don't do anything out of the box themselves and that Drupal is primarily targeted at people that will not program with it, however it seems the overhead for even an experienced programmer to pick up and start running with Drupal is still too high and the patterns you need to help you navigate it too obscured.

What do you think about the Drupal learning curve?

Drupal 7 views vs entityFieldQuery pt1

Introduction

This is the first part of a discussion in which I am hoping to layout a few basic thoughts, get some feedback and then come back with later posts and concrete code examples and design patterns. I am finding it difficult to easily start discussions with other Drupal developers on this topic as it is too easy to slip into a "code everything yourself vs views" discussion. In Drupal 6 Views have become almost engrained as the standard way to solve many problems, some problems that can and should be solved in different ways in Drupal 7 and onward.

Increasingly though I am seeing in comments and posts elsewhere discussion around the new contender for solving some of these problems. A contender that felt like the antidote to the overloading of the 'views metaphor' as soon as I started working with Drupal 7 although I still haven't worked out all the wrinkles. That contender is the EntityFieldQuery.

A very good article was written by Neil Hastings on Building Energy.gov without Views.

I still appreciate views

I am not anti-views, they are very useful, packed full of features, extended by many other modules etc. Often views are the obvious choice for admin tools/reports for users and can be used to give more power to non-technical users. Combined with other modules such as bulk operations or editable fields for example; you can achieve something very quickly that would take a lot of work otherwise.

What I have and do struggle with sometimes is the incredibly complex and 'if it works don't knock it' solutions that can be wrangled out of views. If it takes a fresh developer over an hour trying to work out what is going on with a complex views solutions before they can start debugging a problem, then something is wrong. Once you get beyond a certain point the time you save with simpler views can be quickly lost debugging quirks or behavior that is not what you expect. As you move into contextual filters, relationships etc. there is a lot to learn, using views from within code and the views api, the learning curve can be quite steep.

One thing that views are very good at is a quick route from raw content type to selection and display, there is nothing directly in the EntityFieldQuery that maps to this, so how does it rival views......

Abastraction for data retrieval

EntityFieldQuery rivals views as an abstraction layer for retrieving data. Fundamentally you need to select entity ids (node ids when dealing with node based content) based on some criteria. The database structure Drupal uses doesn't make this very elegant at the database level, tables for each (field and revision of the entity). Selecting your data with sql queries also ties you into a particular storage method.

No code here yet, just a discussion, suppose you have a small content type (a few fields) that needs to undergo some kind of selection process and then get displayed somewhere in a page (perhaps a carousel or similar). A view on the content type that gets displayed as a block would be a typical approach.

Alternatively with very little code you can provide a block with hook_block_info(), specify the block content as the output of a function within hook_block_view(). The function that outputs the content can call on EntityFieldQuery for the node selection and assemble the render array (or return no content if appropriate).

On its own this approach doesn't sound very interesting, however now you have a lot of flexibility to apply randomization, filtering based on taxonomies, caching of the content simply, or basing your cache key on weirdness (that makes sense of your requirements), filters based on things that may be hard to apply in a view, combining selection from multiple queries etc. etc. Maybe drive something from a context using the context module or a cookie the user carries or ... and not worry about whether any of these things integrate with (or can be integrated with) views. You may have to take a simple requirement and add complexity later.

What is more there are lots of ways to customize the output, not least the introduction of custom node view modes in Drupal 7 these can help when building your render array this is key for displaying the different aspects of content in various different ways.

Enterprise level solutions

Ultimately you can define your own entities and extend EntityFieldQuery (it is a PHP class) to meet some of your specific requirements. Defining your own implementation of the EntityFieldControllerInterface or extending the default implementation gives all the potential power you need. Even moving your entity storage to MongoDB

Summary

Although it seems clear that the role of views is (or should be) changing for Drupal 7 onwards, there is not a lot of guidance or clear examples of best practice. I am hoping to generate a little discussion and find out/confirm what the best practice should be.

Break out from Drupal (from the inside)

Quick post: sometimes you (or someone you are working with) may need to break out from Drupal, and generate a special page perhaps based on some existing php code that has been adapted or similar, but that is now pulling data from drupal.

The following code is a brief example of an approach that allows foo.php (in this example) to have full access to any to any variables you may create from querying Drupal and perhaps pick up arguments from the url. foo.php in this case is now responsible for generating the page output.

The use of drupal_exit(); in this case still allow modules to implement exit hooks.

/**
 * Implements hook_menu()
 */
function my_module_menu() {
 
  $items = array();
 
  $items['foo'] = array(
    'title' => t("Foo"),
    'type' => MENU_NORMAL_ITEM,
    'page callback' => 'my_module_foo_page',
    'page arguments' => array(),
  );
 
  return $items;
}

/**
 * My module foo page callback
 */
function my_module_foo_page($arg1 = false, $arg2 = false, $arg3 = false) {
  
  //do some regular Drupal stuff here, populate some variable etc. 

  drupal_add_http_header('Content-Type','text/html');
  include_once('foo.php');
  drupal_exit();
}

Welcome new hook_module_implements_alter

Introduction

Whilst trying to debug something in Drupal 7, I came across a nice new hook a while back. hook_module_implements_alter allows you to hook into any calls to module_implements. You get to find out which hook is being called, and as a bonus to you get to see which modules are returning implementations for the hook and block them if you so wish.

Why is this useful?

I have to admit that this doesn't appear immediately useful, however Jamie has already described very well how you can use this hook. In very quick summary if a module is doing something in a hook that you don't want it to do, you can intervene, you can step in block it and if necessary provide your own implementation, the post I have linked to provides a good explanation of this.

There are other ways in which this hook may be useful, brutally simple to start with is just to quickly find out which hooks are being called for a page request and which modules are returning implementations for them.

function {mymodule}_module_implements_alter(&$implementations, $hook) {  
  drupal_set_message("<pre>" . print_r($hook,true) . "</pre>");
  drupal_set_message("<pre>" . print_r($implementations,true) . "</pre>");
}

This will naturally print a long list, which can be useful in emergencies, this hook could be used for more sophisticated debugging.

Drupal taxonomy manager module

Introduction

I am going to start using this blog to record more technical stuff and ideas, particularly Drupal related, sometimes just random thoughts or comments sometimes, more heavyweight. Hopefully if I tag well and re-organise the blog it should be possible to keep track of various disparate threads and ideas.

Drupal admin is still somewhat awkward in areas, especially in areas like menus and taxonomy where an incredible amount of functionality and a strong api is hampered by administration difficulties in some cases. Managing taxonomies in the administration interface is prime example.

The Taxonomy manager module is a terrific administration module that brings the regular administration of taxonomies forwards in leaps and bounds.

The module

The module has worked well for me in two different scenarios so far, firstly in populating and administrating taxonomies it is far far easier to use than the default Drupal administration. In a few cases I have been able to use this module where I would normally consider populating a taxonomy programmatically. Secondly in the case of a programmatically populated taxonomy that had a huge number of entries normal Drupal admin crashed on display in my web-browser. The taxonomy manager module gracefully handled the display with paging.

Programmer working hours (is 9-5 enough?)

This is an old post pulled from another blog for preservation

Introduction

A while ago I heard the phrase "if they want to work 9-5 then why not just work in a shop?" applied to programmers, the implication being that programming is not a 9-5 kind of job. I have been thinking about this on and off and discussing with people when I get the opportunity, apart from the fact that many people in shops work very long hours there are a number of problems with what was being suggested here.

My thoughts

Computer programming is undoubtedly one of those professions where people are often expected to work long hours, usually without extra reward. A major problem is that we often don't know for sure how long something is going to take (more on why this is in a later post). Another problem is that there are many times where there is a shortage of staff. So many times programmers are expected to work long hours to address these two problems, and many programmers are prepared to put the time in. Sometimes however this doesn't work out, sometimes the programmer who is happily prepared to pull the occasional all-nighters in company X leaves company Y because of long working hours, what is the difference?
Here is one observation or possible reason: programmers like to feel they have some autonomy over what they are producing, they like to understand the reason they doing something and feel that it makes sense. Programmers don't like repeating their work for no good reason, spending long hours working on something that experience tells them will never be used or repeatedly patching and fixing something that they know full well (and will happily tell anyone who will listen) needs to be re-factored and repaired properly. Programmers may be resilient to a lot of the aforementioned woe if it occurs within normal working hours (somebody else is effectively footing the bill, and obeying orders is the path of least resistance), but if the wasted work that they cannot avoid means they also have to work extra hours then those hours hurt, that lost weekend bites deep.

My conclusion

I actually think that programming is a job that is hard to achieve with a 9-5 mentality, but that the hours outside the 9-5 are what people put in themselves, the things that programmers do for a hobby or to keep up to date with technology changes. Given the space many programmers will do things themselves and extend their own skills and knowledge in a way that ultimately benefits the companies they are working for. Sometimes programmers given appropriate autonomy will even extend part of what they do at work into this extra space.
Many programmers work in a challenging environment of constant change, it is hard to manage this kind of career on a 9-5 basis. You would assume that a language interpretor who gets paid 9-5 doesn't switch off and never use the foreign language outside of work (if he did it would be hard to maintain the level of skill required), I believe that programming is a similar profession.

MySql Definer problems

This is an old post pulled from another blog for preservation

Recently whilst moving a legacy application, I had to make it more secure. In trying to make the database more secure I ran into one particular error in some of our Java code java.sql.BatchUpdateException: The user specified as a definer ('root'@'%') does not exist

The definer ('root'@'%') is the root of my problem (pardon the pun). Our application was accessing MySql as root user (so a compromised application could take control of all the databases on the server), even worse having root@% as a user in the database means that root access is not just restricted to local access.

Having given the application it's own user and remove root access from anywhere everything was running fine until we hit a page activated a trigger. Definers appear in a number of places in MySql including triggers, stored procedures and views (not sure what if any problems occur with a view if the defined no longer exists but will make a point to find out). My problem was an easy fix there was just one trigger that needed altering.

Although I have not had to yet, next time I run into this problem I would be looking to take an approach more similar to this one somewhere in that post or the comments there should be a working solution there to remove the definers from a MySql dump. I assume (would check) that the default definer if set by MySql would be the user credentials used to load the database again.

Fortunately the problem was spotted on a test server first.

Turning Chinese, no such thing as a standard search?

Short but sweet, this one. I have already noticed the filtering bubble on one or two sites that I use, particularly I increasingly get the "Chinese influence". Because of my interest in Chinese language, language learning related things and particularly Chinese related things and people I get increasingly connected to Chinese things and Chinese people. This is actually starting to become a hindrance, from the point of view of some services the granularity of what I may be interested in Chinesewise is pretty low. For example I can usually work out why Linkedin is suggesting I want to link with most of its suggestions, however on more than one occasion now it throws in a random Chinese person, who I have not worked with, isn't interested in the same things as me (technology or otherwise) and isn't even geographically that close (although to be fair UK rather than China is somewhat closer than it could be).

How many ways, are decisions about what I see being influenced by what is on my profile? On one site in particular I seem to get a suspiciously high number of adverts regarding hair restoring products and eye-care/laser treatment/optician related links. I don't search for these things, but it wouldn't take much processing to target me based on my follically challenged, bespectacled profile picture.

Once we wrote content for people to read, then wrote content for machines to read, now write content for people that machines may decide they want to read.

The thing I find really interesting however is that, if you are considering search engine optimisation you also increasingly need to consider the profile of the person who is doing the searching, because all search results are not equal. Perhaps even more worryingly increasingly more of our lives are being determined by computer algorithms, we are moving closer to time when algorithms control the world.

Getting around DNS

This is an old post pulled from another site (for preservation)

Jotting this down really, because it is the type of related problem that crops up every now and again and sometimes if my brain is not firing on all cylinders I get stumped for a while.

The problem

Really wanted to check out an open-source project NOW yet svn checkout couldn't resolve the host for the repository. My PC can't find the DNS, other PCs on our network can't find it, it was there yesterday (or rather it was there for me at home).

The solution

First checked that the site was available for some people by checking on http://downforeveryoneorjustme.org/ apparently it is just me (and a few others I guess). Next I need an IP address so searched in Google for sites that give IP address if you know the DNS (because I can't do this for myself right now), first one couldn't find it (down for them also it seems), the second one gave me an IP address(phew) Next problem is that the IP address just takes me to the server for their hosting company, so I need to add an entry in my own hosts file (/etc/hosts) on my Linux system and another entry for the svn. address I was originally trying to use there. Bingo now my PC knows how to find their server and make the request in a way that takes me to the pages and data I want (using the expected host). In the past have also resorted to trying different DNS servers (opendns or googledns) Additional later note: and of course I could always get a command-line on a remote server somewhere (we have a client server in Hong Kong for example) and use that location to do my investigation from.

JavaScript

This is an old post pulled from another site (for preservation)

JavaScript has always been an important programming language for me, I use the term programming language rather than scripting language deliberately. Now I am taking the time to rationalise the technologies that I use and develop in (develop my skills in), I have to say that JavaScript is part of the core. JavaScript has often been associated with the web-browser and treated there as something to add a little bit of visual whiz-bang or tie together a few things that you might want to do in the browser that are not covered in standard HTML. bandwidth, performance, the danger of JavaScript being turned off, the single execution thread ("a script on this page appears to be busy") and other factors have contributed to a general feeling of "not wanting to do too much in JavaScript". These days performance appears to be much less of a problem, Linux on a PC emulator running in your browser for example. The single execution thread blocking problem is solved with HTML5 web workers. Various libraries from JQuery to Extjs allow you to create an interface entirely in JavaScript. You can build sophisticated mobile applications using Phonegap and Titanium. I feel that as more people develop for HTML5 factors like the scriptable video and SVG and going to drive a lot more development in JavaScript and of-course applications like node.js are continue to raise the profile of JavaScript in other areas. Something that has me really excited though and something I will definitely want to play with when I get a nice stretch of time is Quake2 in JavaScript. This project is based on Jake2 which was a Java port of the open sourced Quake2 code that did leave me with mixed feelings (the code very much looked like a conversion from C to Java mostly via implementing a bunch of static classes), however a long time ago I had some fun writing a mod and experimenting with the original Quake source code. The project looks like it needs a bit of effort to build but I am looking forward to enjoying that at some point.