I discuss some of the major conceptual and methodological problems that have arisen in our attempts to assess the relative levels of farm animal welfare in different housing systems. In some cases these problems arise because applied research has not kept pace with more fundamental research. First, the concepts of animal welfare used by researchers are often too limited and do not address many of the issues of concern to the public. This is particularly true for concepts of animal welfare that ignore the topic of suffering. Redefining animal welfare to make it more convenient for research risks making research irrelevant to the issues at hand. Concepts of animal welfare have not dealt adequately with the multidimensional nature of animal welfare: when housing systems are compared, different welfare indicators favour different housing systems and there is a trade-off between different challenges to animal welfare, but conceptual solutions to this problem are few. Furthermore, we have tended to rely too much on physiological, immune and behavioural measures of welfare that have not adequately been validated, and have not given sufficient weighting to health problems, which are some of the major threats to farm animal welfare. Second, we have focused too much upon the type of housing and have paid less attention to other important sources of variability in animal welfare, especially the quality of stockmanship, nutritional effects and the effects of breeding. Third, we have relied too much on a controlled, experimental approach and have not taken enough advantage of the power of epidemiological approaches to identify the main threats to welfare. Finally, we have tried to use physiological, immune and behavioural measures of welfare before we have adequately understood their underlying biological mechanisms.