SPECTRE
(CLINICAL)

Spectre - a Practical and Educational Clinical Trials Reporting Engine
  Latest Updates ---- Bugs+changes ---- Macro list ---- Script list ---- F.A.Q. ---- Download
  Page Loads
Search this site

powered by FreeFind

Updated: 17 Jun 2015

(If you landed on this page doing general web browsing and are not interested in clinical reporting systems then you might like to look at the sas macro page here. If you are looking for clinical reporting macros then there are two macros webbed here named %unistats and %npcttab. %unistats has its own Powerpoint slide show here. There is a lot of practical Unix stuff that covers both Unix commands and shell scripting that you can link to here and there are a lot of shell script examples you can link to here. Thanks for your visit)

Introduction

This web site contains a complete clinical reporting engine that uses macros written using SAS software. It is called an "engine", rather than the more familiar "clinical reporting system" because although the macros can be used as they are for common reporting needs, the macros are primarily intended to be used "under the bonnet" or "behind the scenes" by SAS programmers with the programmers building an interface for the user that is more friendly and tailored to their reporting needs.

The Spectre macros are very efficient and are designed to cope with huge volumes of data. They use techniques for efficient data analysis that you can read about here.

If you have limited time and want the briefest possible summary of the Spectre reporting system then use the link below.
Spectre in a Nutshell
 

Disclaimer

All software comes with a disclaimer and the same is true for both parts of Spectre. When you install software you have to agree to the terms of using it to the effect that you should ensure the software is suitable for your purposes and that no liability is excepted if the software goes wrong. The macros on this web site are very stable but bugs are found from time to time. Also, you should be aware of what the macros are doing and be particularly careful abiout the statistical values you might choose to show that come out of these reporting macros. Check these values. If a value looks wrong then the chances are it is wrong. It is up to you to ensure what is being reported at least makes sense and looks reasonable. So first, an important disclaimer, and that is....

DO NOT ASSUME THE CODE ON THIS WEB SITE IS "VALIDATED" FOR YOUR PURPOSES

If you intend to use this code for clinical reporting purposes, rather than just for education, then you can only treat what you find here as a validated system if you have validated it yourself, in accordance with industry regulations and your own Standard Operating Procedures. All of the reporting macros have been validated by one of the known users but they have validated it for their own particular mode of use. This does not make it validated for your purposes. Although I have a great deal of confidence in the code members webbed here, having used them for years and knowing that most code members run live, you should not assume, without thorough checking, that they will work for you in the way you expect or be suitable for your needs. To stress this point, every single code member will carry a disclaimer in the header to remind you of this. No code members should be used unless you fully understand and accept the disclaimer in the header.

Download

You can download the Spectre system from the page you can link to below (the "download" page) as a number of zip files. You will probably just be interested in the clinical reporting macros but these also rely on a library of macros that are the utility macros so if you are only interested in the Clinical Reporting macros then you must download the utility macros as well. You need to make both libraries available and place both libraries on your sasautos path for the Clinical reporting macros to work.

download
 

The origins of Spectre

Spectre is based on an old Unix-based clinical reporting system that was once used at Glaxo Wellcome before they merged with SmithKline Beecham to form GSK. This old clinical reporting system was phased out after the merger and replaced with a new system. I was working at the company at the time and I preferred the older system because, by accident and not design, it was structured in a way that it was just a few steps short of achieving the ultimate goal of a reporting system, which is to be a "push button" reporting system (explained later). I redesigned the old system and simplified it to better put it in a position to achieve later status of being a "push button" system.

The components of a "reporting engine"

For me, at least, a "reporting engine" has a few identifiable components. There are many things that could be added to the above list that people would think are essential but if they have these components in place then I would say they have a "reporting engine" and the "core" of a reporting system. A reporting "system" will have a fuller set of macros for producing reports and some of these macros will call one of the two core macros described above. Quite often, reporting systems have sophisticated, web-based front ends. There are no plans to make a front end available as yet.

Why you need a reporting system

All pharmaceutical companies want to be efficient in producing clinical reports to save on costs. For the large pharmaceutical companies, perhaps with hundreds of programmers worldwide, they know that they can save a lot of programming time by having "standard macros" to do most of the reporting. Maybe 25% or more of programmer time can be saved. That comes to a lot of potential money saved, so the larger pharmaceutical companies will invest whatever it takes to build a reporting system. And if they have enough programmers to select from, they can find the skilled people to achieve this. It is not so easy for smaller companies or CROs to create such a reporting system as they have fewer skilled programmers to choose from and whatever investment they make to build such a system will be a disproportionally higher proportion of their programming costs.

There are other reasons for having such a reporting system. Reporting systems can help with presentation in that they impose a consistency in layout style. They can help with reliability and accuracy of safety reporting using the two major reporting macros as described above, since these macros, through their thorough usage and careful monitoring and maintenance, will arrive at the point where they are completely trustworthy and so their output will require only minimal checking. A good system will ensure all programs are run in the correct order and that a record is kept of who ran what and when. Also, that a list of outputs created is recorded somewhere.

The ultimate goal of a reporting system

The ultimate goal of a clinical reporting system is to produce reports automatically from clinical data. Now if you thought that last sentence was clear, I'm going to disappoint you by qualifying what I mean by "clinical data". This "clinical data" might not be the same as the data collected for a clinical trial. The structure might have changed as well as the naming conventions and other values might have been derived and merged in with the data to make "derived datasets" or "reporting-ready datasets" and it is that to which I refer when I write "clinical data". If this data exists in a standard form (which might be the case in the future as a result of the CDISC project) and the type of analysis is decided in advance, then it should be possible to generate reports automatically - just by pushing a button, maybe. Achieving this might take many years from the date of my writing this and indeed, it may never be achieved, but if this goal is constantly striven for, greater efficiencies will be realized along the way. Spectre embraces this ultimate goal of being a "push-button" reporting system. Short calls to reporting macros can be held in the same place that titles and footnotes are defined and that can lay the foundation for an entirely automated reporting system. It has already been used in this way to generate all the tables for one clinical study except it fell short in that the macros it called to produce the reports were study specific, since the data structure and variable names varied from study to study. But when the data structures are harmonized then that ultimate goal should be achieved.

Spectre "Extras"

When Spectre was first implemented, the company was entering into a merger. It was a requirement that the reporting system could produce output in the current style as well as a future style (yet to be defined) and that both styles of reports would be recreatable. So the system had to have the capability of producing output in more than one client style. Though not written with CROs in mind, this will be of obvious benefit to them since they work for multiple clients. It also makes Spectre a candidate for becoming a "standard" reporting engine for use across the industry.

Safety vs. Efficacy Reporting

The Spectre Reporting Macros do both safety and efficacy reporting in the same report in the way you specify. The reporting macros have powerful statistical capabilities and huge flexibility to allow you to include efficacy statistics into the report from whatever source and in any position you specify. You just have to know the "proc glm" or "proc freq" options to specify, what ods tables are created as a result and what variables in those ods tables to use and you can place them whereever you want without need for further programming. For simple p-values, it is a lot easier.

The User Guides

I hope you found the above introduction to Spectre interesting. Hopefully, you will have caught a glimpse that this is a reporting engine with purpose and direction. The next step is to read the User Guides that can be linked to below. Remember that the Reporting Macros are independent of the System macros. I have flagged the "system" topics below in brackets following the page title so that you can skip them if you want to. You should read the pages in the order listed.

Building the derived datasets (system)

Study Reporting Process Flow (system)

Working in batch and creating titles (system)

%titles (system)

%openrep and %closerep (system)

%popfmt

Creating Test Datasets

%unistats PowerPoint demonstration

%unistats

%unistats "freqsept" parameters for efficacy reporting

%unistats with "by" variables

Using %unistats with Lab data (1)

Using %unistats with Lab data (2)

Using %unistats with Lab data (3)

Using %unistats with Lab data (4)

Miscellaneous Lab macros  new

Reporting Treatment-Emergent Adverse Events  new

Use of Temporary Treatment Variables new

%npcttab  updated 01 June 2015

%aetab

More than Ten Footnotes

%unistats and %npcttab for MS Windows users

Validation programming

Spectre Fun Quiz 1

Ad-hoc Reporting and PDFs (system)

Legacy systems and PDFs (system)

Printing Reports (system)

Power Reporting using Spectre (system)

%xytitles - an example client titles macro (system)

General Standards for Clinical Reports

Writing Reporting Macros for Spectre

Spectre System Administration

If you are a programmer then you will not need to know about Spectre administration. The administrator has extra tasks to do. Before you start working on a study, the administrator will have set up the file "protocol.txt" in your programs folder and they will have put the file "titles.template" there which you can copy and use as the template for your titles members. They will likely be the person to run the suite of programs and to create the PDFs using the utilities provided. If the suite of programs is large and the amount of output large then the administrator will have to split the file "donelist.txt" (the list of all output produced) into "donelist sections" and to create a PDF from each section. And when the PDFs have been created, they will be the person to run the "printpdfbookmarks" utility to extract the list of PDF bookmarks from the PDF files. If Spectre is being used to create the derived datasets then the administrator will check the naming convention of the programs to ensure that the suite of programs runs in the correct order and to ask programmers to rename some of the programs if this is not the case. They will likely check the naming convention used for the programs that produce output and sometimes programmers will be asked to rename their programs (there are utilities to help with this described on the "Utility Scripts" page). So if you are a programmer, read the administrators guide if you are interested, but you will find more to interest you in the "Scripts and Macros" section.

Spectre Administration

Scripts and Macros

If you are a programmer using Spectre then after a while you will understand that it is aimed at increasing speed and efficiency. It is there to remove barriers to greater productivity, not to add them. Greater speed and efficiency means that single programmers will maybe end up with a hundred programs or more. But having a hundred programs or more has the potential of turning into an administrative nightmare. It would be like winning the technical battle of efficient coding only to be overwhelmed by sheer weight of numbers. That would be very stressful for a programmer but help is at hand. Spectre has many utility scripts to help the programmer manage his hundred or more programs written using SAS software and it is important for the programmer to be aware of these scripts and how they can help. Without knowledge of these scripts, a programmer will eventually hit the barrier caused by sheer weight of numbers. But knowing that these scripts exist and knowing how to use them, having hundreds of programs should not be a problem. It will take almost none of your time and concentration to deal with it.

Here is a typical example of having to divert your attention to work on one of your hundreds of  programs and how you would cope with it. Suppose someone tells you that they would like the footnote to be changed for Table 13.4.5.1 in study XYZ. There is no need to look up this table in a spreadsheet somewhere. You go to one of your terminal windows, you probably have variables set up in your ".bashrc" member (sometimes it is called ".bashrc_own") to help you jump to directories. So you type in the command "cd $xyzp" (or something like that) to take you to the study XYZ programs directory. You then type in the command "wt 13.4.5.1" to tell you what program it is. You then set "prog" equal to this program and then (assuming "prog" has already been "exported") you type in the command "titles" and you have the titles member opened in edit mode. You then change the footnote, recreate the titles members with the "crtitlesds" command (you may even have an alias for this that is shorter. I use "crt" as an alias for this). Then when this completes you type in "sasb". You see that there are no errors or warnings so you type in "lis" to look at the output. You see that the new footnote is fine so you close the window and reset the current directory to what it was before using the command "cd -", and it is done. You are back working on what you were doing before after only a few seconds have passed. You hardly noticed the intrusion and now you have even forgotten about it.

If you know these scripts well and are familiar with Unix commands then an interruption, such as the one described above, will not break your concentration and you will find it easy to cope with hundreds of programs. As for writing programs using SAS software, you will find many macros available to you, linked to from these pages, to do common difficult tasks. They will help you to write portable macros that can be used in multiple studies. After a while, you will find that you are writing less and less code and instead, copying macros and titles from study to study. There are scripts to help you do this. After you copy them across, there are scripts to fix the study, drug and increment names in the headers. There are scripts to help you do a mass renaming of files. There are scripts to help you tidy up your ".log" and ".lst" files that get left over after a mass renaming has been done. If you take the trouble to learn about these scripts and macros then you will easily be able to cope with having hundreds of programs. Hopefully, you will you become one of the most highly productive clinical reporting programmers in the field and yet be stress free with it.

The most important macros are %titles, %openrep, %closerep, %popfmt, %unistats and %npcttab. These have their own pages that you can link to above and you should make sure you read them thoroughly. The "macros" page you can link to below just contains summary information about these important macros. As for the scripts, the ones the programmer will be most interested in are the "Utility Scripts", as those are the ones that will help speed and organize their work.

Scripts

Macros

Spectre at Work

It is important to realise that Spectre is a very simple reporting system. It is minimilistic and intended that way. It makes available a huge amount of functionality (particularly with the reporting macros) without expanding its role and forcing you to change the way you work with it. Instead, it works for you. Used as a back-end system in the way described here, it is likely far simpler than any other reporting system currently in use at any major pharmaceutical company. Many front-end systems become over-complicated, difficult to use and slow over time, hugely reducing productivity. But Spectre will always remain as as a powerful, efficient and fast back-end system. It has a lot of components, as you will see if you look, but then so does any motorised utility that does something simple like help you cut your grass. Using Spectre is logical and simple. To help you realise that, you can link to the page below and see a very simple dummy study, with just two derived (dummy) datasets and two tables, reported from empty directories through to final report run, showing all the administrative tasks performed. Finally, the transformation to get it ready for a "push buttion" reporting system is demonstrated and explained. If you wanted to skip through it quickly, it might only take you a few minutes.

Spectre at Work

Other clinical macros

I have included three other topics with their own macros that were not part of the original Spectre as I am sure you will have a use for them occasionally.

Time-to-Event analysis

Merging with a dose

LOCF (Last Observation Carried Forward) processing

Some SAS techniques

In creating Spectre, I have used some techniques with sas that not all sas programmers will be familiar with. Without using these techniques, I could never have written Spectre in the time I did. All the techniques, as they relate to writing a reporting system, will be explained in the following document you can link to.

SAS techniques

Learning Unix

As programmers using Spectre become more proficient, they can find that they are responsible for hundreds of programs at a time. Dealing with this many files is easier if you have a good knowledge of Unix commands and you know how to combine them. This is a higher level of skill than most sas programmers would have who work on a Unix platform so there are documents on this web site to help you increase your Unix skills.

Learning Unix

Installing Spectre

If you just want to use the %npcttab and %unistats macros but do not want to set up the whole of Spectre (Clinical) then installing it is easy. All you need to do is download the clinmacros and utilmacros and put them in their own libraries defined to the sasautos path. The rest of the information you can read here is to do with a full Spectre System install for those users planning to set up a full clinical reporting system.

Spectre will work best on a Linux or Unix platform with a proper Linux or Unix version of SAS software (i.e. supporting the -stdio option). Although it has been ported over to Windows you will have to use the Cygwin Linux emulator to run the Spectre scripts. On a Windows platform the scripts to convert output to Postscript files and thence to PDFs will not work as to get this working would require a lot of disk space and a lot of work that is probably not needed. If you are a CRO then it is unlikely the client will ask for your output in PDF format. If they want it in that format then it is easier if the client accepts your text output files as deliverables and does the conversion to a PDF themselves. Apart from Postscript files and PDFs, the rest will work on a Windows platform with Cygwin installed. It will work fine with SAS Learning Edition 4.1 buth this will limit the number of observations you can read so you can not use them for production work.

The full installation of Spectre (Clinical) is quite complex so there are guides for this that you can link to below.

Installing Spectre on Linux or Unix

Installing Spectre on Windows

Key Components of Spectre System

Spectre System has only a few key components. So long as those components are not changed, there is huge scope to tailor it so that it works in the way you want it to. I do not want to give the impression that Spectre System is a fixed system. It is not. What you see on this web site is more a full-blown example of what it can do. You can change Spectre System in many ways to suit, once you have a feel for what these key components are. This is explained on the following page.

Key Components of Spectre

Amending Spectre System

Once you know the key components of Spectre System it should put you in a position of being able to amend Spectre to suit your site standards. This is explained on the following page.

Amending Spectre

Terms for Spectre Support

The Spectre Clinical reporting macros require a licence for use as of 2012. The rest is unsupported and free. The licence fee is described on the download page.

Conclusion

Everything you need to know about Spectre should be documented on this web site. A few unrelated topics are included in the FAQ which you can link to below. I hope you will find Spectre fast to learn, easy to use and that it will increase productivity several fold.

F.A.Q.

contact the author





























SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.