Drupal 7 views vs entityFieldQuery pt1

Introduction

This is the first part of a discussion in which I am hoping to layout a few basic thoughts, get some feedback and then come back with later posts and concrete code examples and design patterns. I am finding it difficult to easily start discussions with other Drupal developers on this topic as it is too easy to slip into a "code everything yourself vs views" discussion. In Drupal 6 Views have become almost engrained as the standard way to solve many problems, some problems that can and should be solved in different ways in Drupal 7 and onward.

Increasingly though I am seeing in comments and posts elsewhere discussion around the new contender for solving some of these problems. A contender that felt like the antidote to the overloading of the 'views metaphor' as soon as I started working with Drupal 7 although I still haven't worked out all the wrinkles. That contender is the EntityFieldQuery.

A very good article was written by Neil Hastings on Building Energy.gov without Views.

I still appreciate views

I am not anti-views, they are very useful, packed full of features, extended by many other modules etc. Often views are the obvious choice for admin tools/reports for users and can be used to give more power to non-technical users. Combined with other modules such as bulk operations or editable fields for example; you can achieve something very quickly that would take a lot of work otherwise.

What I have and do struggle with sometimes is the incredibly complex and 'if it works don't knock it' solutions that can be wrangled out of views. If it takes a fresh developer over an hour trying to work out what is going on with a complex views solutions before they can start debugging a problem, then something is wrong. Once you get beyond a certain point the time you save with simpler views can be quickly lost debugging quirks or behavior that is not what you expect. As you move into contextual filters, relationships etc. there is a lot to learn, using views from within code and the views api, the learning curve can be quite steep.

One thing that views are very good at is a quick route from raw content type to selection and display, there is nothing directly in the EntityFieldQuery that maps to this, so how does it rival views......

Abastraction for data retrieval

EntityFieldQuery rivals views as an abstraction layer for retrieving data. Fundamentally you need to select entity ids (node ids when dealing with node based content) based on some criteria. The database structure Drupal uses doesn't make this very elegant at the database level, tables for each (field and revision of the entity). Selecting your data with sql queries also ties you into a particular storage method.

No code here yet, just a discussion, suppose you have a small content type (a few fields) that needs to undergo some kind of selection process and then get displayed somewhere in a page (perhaps a carousel or similar). A view on the content type that gets displayed as a block would be a typical approach.

Alternatively with very little code you can provide a block with hook_block_info(), specify the block content as the output of a function within hook_block_view(). The function that outputs the content can call on EntityFieldQuery for the node selection and assemble the render array (or return no content if appropriate).

On its own this approach doesn't sound very interesting, however now you have a lot of flexibility to apply randomization, filtering based on taxonomies, caching of the content simply, or basing your cache key on weirdness (that makes sense of your requirements), filters based on things that may be hard to apply in a view, combining selection from multiple queries etc. etc. Maybe drive something from a context using the context module or a cookie the user carries or ... and not worry about whether any of these things integrate with (or can be integrated with) views. You may have to take a simple requirement and add complexity later.

What is more there are lots of ways to customize the output, not least the introduction of custom node view modes in Drupal 7 these can help when building your render array this is key for displaying the different aspects of content in various different ways.

Enterprise level solutions

Ultimately you can define your own entities and extend EntityFieldQuery (it is a PHP class) to meet some of your specific requirements. Defining your own implementation of the EntityFieldControllerInterface or extending the default implementation gives all the potential power you need. Even moving your entity storage to MongoDB

Summary

Although it seems clear that the role of views is (or should be) changing for Drupal 7 onwards, there is not a lot of guidance or clear examples of best practice. I am hoping to generate a little discussion and find out/confirm what the best practice should be.

Break out from Drupal (from the inside)

Quick post: sometimes you (or someone you are working with) may need to break out from Drupal, and generate a special page perhaps based on some existing php code that has been adapted or similar, but that is now pulling data from drupal.

The following code is a brief example of an approach that allows foo.php (in this example) to have full access to any to any variables you may create from querying Drupal and perhaps pick up arguments from the url. foo.php in this case is now responsible for generating the page output.

The use of drupal_exit(); in this case still allow modules to implement exit hooks.

/**
 * Implements hook_menu()
 */
function my_module_menu() {
 
  $items = array();
 
  $items['foo'] = array(
    'title' => t("Foo"),
    'type' => MENU_NORMAL_ITEM,
    'page callback' => 'my_module_foo_page',
    'page arguments' => array(),
  );
 
  return $items;
}

/**
 * My module foo page callback
 */
function my_module_foo_page($arg1 = false, $arg2 = false, $arg3 = false) {
  
  //do some regular Drupal stuff here, populate some variable etc. 

  drupal_add_http_header('Content-Type','text/html');
  include_once('foo.php');
  drupal_exit();
}

Welcome new hook_module_implements_alter

Introduction

Whilst trying to debug something in Drupal 7, I came across a nice new hook a while back. hook_module_implements_alter allows you to hook into any calls to module_implements. You get to find out which hook is being called, and as a bonus to you get to see which modules are returning implementations for the hook and block them if you so wish.

Why is this useful?

I have to admit that this doesn't appear immediately useful, however Jamie has already described very well how you can use this hook. In very quick summary if a module is doing something in a hook that you don't want it to do, you can intervene, you can step in block it and if necessary provide your own implementation, the post I have linked to provides a good explanation of this.

There are other ways in which this hook may be useful, brutally simple to start with is just to quickly find out which hooks are being called for a page request and which modules are returning implementations for them.

function {mymodule}_module_implements_alter(&$implementations, $hook) {  
  drupal_set_message("<pre>" . print_r($hook,true) . "</pre>");
  drupal_set_message("<pre>" . print_r($implementations,true) . "</pre>");
}

This will naturally print a long list, which can be useful in emergencies, this hook could be used for more sophisticated debugging.

Drupal taxonomy manager module

Introduction

I am going to start using this blog to record more technical stuff and ideas, particularly Drupal related, sometimes just random thoughts or comments sometimes, more heavyweight. Hopefully if I tag well and re-organise the blog it should be possible to keep track of various disparate threads and ideas.

Drupal admin is still somewhat awkward in areas, especially in areas like menus and taxonomy where an incredible amount of functionality and a strong api is hampered by administration difficulties in some cases. Managing taxonomies in the administration interface is prime example.

The Taxonomy manager module is a terrific administration module that brings the regular administration of taxonomies forwards in leaps and bounds.

The module

The module has worked well for me in two different scenarios so far, firstly in populating and administrating taxonomies it is far far easier to use than the default Drupal administration. In a few cases I have been able to use this module where I would normally consider populating a taxonomy programmatically. Secondly in the case of a programmatically populated taxonomy that had a huge number of entries normal Drupal admin crashed on display in my web-browser. The taxonomy manager module gracefully handled the display with paging.

Programmer working hours (is 9-5 enough?)

This is an old post pulled from another blog for preservation

Introduction

A while ago I heard the phrase "if they want to work 9-5 then why not just work in a shop?" applied to programmers, the implication being that programming is not a 9-5 kind of job. I have been thinking about this on and off and discussing with people when I get the opportunity, apart from the fact that many people in shops work very long hours there are a number of problems with what was being suggested here.

My thoughts

Computer programming is undoubtedly one of those professions where people are often expected to work long hours, usually without extra reward. A major problem is that we often don't know for sure how long something is going to take (more on why this is in a later post). Another problem is that there are many times where there is a shortage of staff. So many times programmers are expected to work long hours to address these two problems, and many programmers are prepared to put the time in. Sometimes however this doesn't work out, sometimes the programmer who is happily prepared to pull the occasional all-nighters in company X leaves company Y because of long working hours, what is the difference?
Here is one observation or possible reason: programmers like to feel they have some autonomy over what they are producing, they like to understand the reason they doing something and feel that it makes sense. Programmers don't like repeating their work for no good reason, spending long hours working on something that experience tells them will never be used or repeatedly patching and fixing something that they know full well (and will happily tell anyone who will listen) needs to be re-factored and repaired properly. Programmers may be resilient to a lot of the aforementioned woe if it occurs within normal working hours (somebody else is effectively footing the bill, and obeying orders is the path of least resistance), but if the wasted work that they cannot avoid means they also have to work extra hours then those hours hurt, that lost weekend bites deep.

My conclusion

I actually think that programming is a job that is hard to achieve with a 9-5 mentality, but that the hours outside the 9-5 are what people put in themselves, the things that programmers do for a hobby or to keep up to date with technology changes. Given the space many programmers will do things themselves and extend their own skills and knowledge in a way that ultimately benefits the companies they are working for. Sometimes programmers given appropriate autonomy will even extend part of what they do at work into this extra space.
Many programmers work in a challenging environment of constant change, it is hard to manage this kind of career on a 9-5 basis. You would assume that a language interpretor who gets paid 9-5 doesn't switch off and never use the foreign language outside of work (if he did it would be hard to maintain the level of skill required), I believe that programming is a similar profession.

MySql Definer problems

This is an old post pulled from another blog for preservation

Recently whilst moving a legacy application, I had to make it more secure. In trying to make the database more secure I ran into one particular error in some of our Java code java.sql.BatchUpdateException: The user specified as a definer ('root'@'%') does not exist

The definer ('root'@'%') is the root of my problem (pardon the pun). Our application was accessing MySql as root user (so a compromised application could take control of all the databases on the server), even worse having root@% as a user in the database means that root access is not just restricted to local access.

Having given the application it's own user and remove root access from anywhere everything was running fine until we hit a page activated a trigger. Definers appear in a number of places in MySql including triggers, stored procedures and views (not sure what if any problems occur with a view if the defined no longer exists but will make a point to find out). My problem was an easy fix there was just one trigger that needed altering.

Although I have not had to yet, next time I run into this problem I would be looking to take an approach more similar to this one somewhere in that post or the comments there should be a working solution there to remove the definers from a MySql dump. I assume (would check) that the default definer if set by MySql would be the user credentials used to load the database again.

Fortunately the problem was spotted on a test server first.

Turning Chinese, no such thing as a standard search?

Short but sweet, this one. I have already noticed the filtering bubble on one or two sites that I use, particularly I increasingly get the "Chinese influence". Because of my interest in Chinese language, language learning related things and particularly Chinese related things and people I get increasingly connected to Chinese things and Chinese people. This is actually starting to become a hindrance, from the point of view of some services the granularity of what I may be interested in Chinesewise is pretty low. For example I can usually work out why Linkedin is suggesting I want to link with most of its suggestions, however on more than one occasion now it throws in a random Chinese person, who I have not worked with, isn't interested in the same things as me (technology or otherwise) and isn't even geographically that close (although to be fair UK rather than China is somewhat closer than it could be).

How many ways, are decisions about what I see being influenced by what is on my profile? On one site in particular I seem to get a suspiciously high number of adverts regarding hair restoring products and eye-care/laser treatment/optician related links. I don't search for these things, but it wouldn't take much processing to target me based on my follically challenged, bespectacled profile picture.

Once we wrote content for people to read, then wrote content for machines to read, now write content for people that machines may decide they want to read.

The thing I find really interesting however is that, if you are considering search engine optimisation you also increasingly need to consider the profile of the person who is doing the searching, because all search results are not equal. Perhaps even more worryingly increasingly more of our lives are being determined by computer algorithms, we are moving closer to time when algorithms control the world.

Getting around DNS

This is an old post pulled from another site (for preservation)

Jotting this down really, because it is the type of related problem that crops up every now and again and sometimes if my brain is not firing on all cylinders I get stumped for a while.

The problem

Really wanted to check out an open-source project NOW yet svn checkout couldn't resolve the host for the repository. My PC can't find the DNS, other PCs on our network can't find it, it was there yesterday (or rather it was there for me at home).

The solution

First checked that the site was available for some people by checking on http://downforeveryoneorjustme.org/ apparently it is just me (and a few others I guess). Next I need an IP address so searched in Google for sites that give IP address if you know the DNS (because I can't do this for myself right now), first one couldn't find it (down for them also it seems), the second one gave me an IP address(phew) Next problem is that the IP address just takes me to the server for their hosting company, so I need to add an entry in my own hosts file (/etc/hosts) on my Linux system and another entry for the svn. address I was originally trying to use there. Bingo now my PC knows how to find their server and make the request in a way that takes me to the pages and data I want (using the expected host). In the past have also resorted to trying different DNS servers (opendns or googledns) Additional later note: and of course I could always get a command-line on a remote server somewhere (we have a client server in Hong Kong for example) and use that location to do my investigation from.

JavaScript

This is an old post pulled from another site (for preservation)

JavaScript has always been an important programming language for me, I use the term programming language rather than scripting language deliberately. Now I am taking the time to rationalise the technologies that I use and develop in (develop my skills in), I have to say that JavaScript is part of the core. JavaScript has often been associated with the web-browser and treated there as something to add a little bit of visual whiz-bang or tie together a few things that you might want to do in the browser that are not covered in standard HTML. bandwidth, performance, the danger of JavaScript being turned off, the single execution thread ("a script on this page appears to be busy") and other factors have contributed to a general feeling of "not wanting to do too much in JavaScript". These days performance appears to be much less of a problem, Linux on a PC emulator running in your browser for example. The single execution thread blocking problem is solved with HTML5 web workers. Various libraries from JQuery to Extjs allow you to create an interface entirely in JavaScript. You can build sophisticated mobile applications using Phonegap and Titanium. I feel that as more people develop for HTML5 factors like the scriptable video and SVG and going to drive a lot more development in JavaScript and of-course applications like node.js are continue to raise the profile of JavaScript in other areas. Something that has me really excited though and something I will definitely want to play with when I get a nice stretch of time is Quake2 in JavaScript. This project is based on Jake2 which was a Java port of the open sourced Quake2 code that did leave me with mixed feelings (the code very much looked like a conversion from C to Java mostly via implementing a bunch of static classes), however a long time ago I had some fun writing a mod and experimenting with the original Quake source code. The project looks like it needs a bit of effort to build but I am looking forward to enjoying that at some point.

Moving some posts in here

I recently started posting some random thoughts, comments etc, mostly about techie stuff else where. i think they are better posted here so I will drag those four articles into this blog shortly

Again the purpose of this blog is mainly just to collect together things I might want to come back to, being online it is accessible anywhere, and although nobody else reads it, it helps me to focus by knowing that in theory they can.

Dropbox, more cross appliance goodness

Another short but sweet, after my last post about the Kindle I reaslized I had forgot to mention dropbox. Definitely worthy of a mention in the same thought as again I have had a wonderful experience from the service. Not only does drop box keep files synced and available for me across a number of devices it also provides extra functionality on my Ipod Touch, I can choose to save files there of various different types, actually open them etc. etc.

Now I am starting to sound like an advert for Kindle and Dropbox, have to think of something a bit gloomier and more negative for my next post.