(If you landed on this page doing general web browsing and are not
interested in clinical reporting systems then you might like to look at
the sas macro page here. If
you are looking for clinical reporting macros then there are two
macros webbed here named %unistats
and %npcttab. %unistats has
its own Powerpoint slide show
here.
There is a lot of practical Unix stuff that covers both Unix commands and
shell scripting that you can link to here
and there are a lot of shell script examples you can link to here.
Thanks for your visit)
Introduction
This web site contains a complete clinical reporting engine
that
uses macros written using SAS software. It is called an "engine",
rather than the more familiar "clinical reporting system" because
although the macros can be used as they are for common reporting needs,
the macros are primarily intended to be used "under the bonnet" or "behind
the scenes" by SAS programmers with the programmers building an interface
for the user that is more friendly and tailored to their reporting needs.
The Spectre macros are very efficient and are designed to cope with
huge volumes of data. They use techniques for efficient data analysis that
you can read about here.
If you have limited time and want the briefest possible summary of
the Spectre reporting system then use the link below. Spectre in a Nutshell
Disclaimer
All software comes with a disclaimer and the same is true for both parts
of Spectre. When you install software you have to agree to the terms of
using it to the effect that you should ensure the software is suitable
for your purposes and that no liability is excepted if the software goes
wrong. The macros on this web site are very stable but bugs are found from
time to time. Also, you should be aware of what the macros are doing and
be particularly careful abiout the statistical values you might choose
to show that come out of these reporting macros. Check these values. If
a value looks wrong then the chances are it is wrong. It is up to
you to ensure what is being reported at least makes sense and looks reasonable.
So first, an important disclaimer, and that is....
DO NOT ASSUME THE CODE ON THIS WEB SITE IS "VALIDATED"
FOR YOUR PURPOSES
If you intend to use this code for clinical reporting purposes, rather
than just for education, then you can only treat what you find here
as a validated system if you have validated it yourself, in accordance
with industry regulations and your own Standard
Operating Procedures. All of the reporting macros have been validated
by one of the known users but they have validated it for their own particular
mode of use. This does not make it validated for your purposes.
Although I have a great deal of confidence in the code members webbed here,
having used them for years and knowing that most code members run live,
you should not assume, without thorough checking, that they will work for
you in the way you expect or be suitable for your needs. To stress this
point, every single code member will carry a disclaimer in the header to
remind you of this. No code members should be used unless you fully understand
and accept the disclaimer in the header.
Download
You can download the Spectre system from the page you can link to below
(the "download" page) as a number of zip files. You will probably just
be interested in the clinical reporting macros but these also rely
on a library of macros that are the utility macros so if you are
only interested in the Clinical Reporting macros then you must download
the utility macros as well. You need to make both libraries available
and place both libraries on your sasautos path for the Clinical
reporting macros to work.
Spectre is based on an old Unix-based clinical reporting system that was
once used at Glaxo Wellcome before they merged with SmithKline Beecham
to form GSK. This old clinical reporting system was phased out after the
merger and replaced with a new system. I was working at the company at
the time and I preferred the older system because, by accident and not
design, it was structured in a way that it was just a few steps short of
achieving the ultimate goal of a reporting system, which is to be a "push
button" reporting system (explained later). I redesigned the old system
and simplified it to better put it in a position to achieve later status
of being a "push button" system.
The components of a "reporting engine"
For me, at least, a "reporting engine" has a few identifiable components.
On the reporting side, it should have the ability to handle the
bulk of safety reporting, and that requires two major macros - one
macro to calculate summary statistics and category counts with percentages
and another macro to calculate unique patient counts and percentages
(and optionally event counts). Most large pharmaceutical companies (and
many smaller ones and CROs) have such macros (Spectre Reporting goes beyond
this need by being able to do efficacy reporting as well).
On the system side, a "titles" (and footnotes) system is
required such that titles and footnotes can be stored outside sas code
members and somehow get included into tables and listings.
A means of running all the programs in the correct order to create
the output.
A means of gathering all the outputs produced in the correct order
or at least telling a separate documentation system about it.
There are many things that could be added to the above list that people
would think are essential but if they have these components in place then
I would say they have a "reporting engine" and the "core" of a reporting
system. A reporting "system" will have a fuller set of macros for
producing reports and some of these macros will call one of the two core
macros described above. Quite often, reporting systems have sophisticated,
web-based front ends. There are no plans to make a front end available
as yet.
Why you need a reporting system
All pharmaceutical companies want to be efficient in producing clinical
reports to save on costs. For the large pharmaceutical companies, perhaps
with hundreds of programmers worldwide, they know that they can save a
lot of programming time by having "standard macros" to do most of the reporting.
Maybe 25% or more of programmer time can be saved. That comes to a lot
of potential money saved, so the larger pharmaceutical companies will invest
whatever it takes to build a reporting system. And if they have enough
programmers to select from, they can find the skilled people to achieve
this. It is not so easy for smaller companies or CROs
to create such a reporting system as they have fewer skilled programmers
to choose from and whatever investment they make to build such a system
will be a disproportionally higher proportion of their programming costs.
There are other reasons for having such a reporting system. Reporting
systems can help with presentation in that they impose a consistency
in layout style. They can help with reliability and accuracy of
safety reporting using the two major reporting macros as described above,
since these macros, through their thorough usage and careful monitoring
and maintenance, will arrive at the point where they are completely trustworthy
and so their output will require only minimal checking. A good system will
ensure
all programs are run in the correct order and that a record
is kept of who ran what and when. Also, that a list of outputs created
is recorded somewhere.
The ultimate goal of a reporting system
The ultimate goal of a clinical reporting system is to produce reports
automatically from clinical data. Now if you thought that last sentence
was clear, I'm going to disappoint you by qualifying what I mean by "clinical
data". This "clinical data" might not be the same as the data collected
for a clinical trial. The structure might have changed as well as the naming
conventions and other values might have been derived and merged in with
the data to make "derived datasets" or "reporting-ready datasets"
and it is that to which I refer when I write "clinical data". If this data
exists in a standard form (which might be the case in the future as a result
of the CDISC project)
and the type of analysis is decided in advance, then it should be possible
to generate reports automatically - just by pushing a button, maybe. Achieving
this might take many years from the date of my writing this and indeed,
it may never be achieved, but if this goal is constantly striven for, greater
efficiencies will be realized along the way. Spectre embraces this ultimate
goal of being a "push-button" reporting system. Short calls to reporting
macros can be held in the same place that titles and footnotes are defined
and that can lay the foundation for an entirely automated reporting system.
It has already been used in this way to generate all the tables for one
clinical study except it fell short in that the macros it called to produce
the reports were study specific, since the data structure and variable
names varied from study to study. But when the data structures are harmonized
then that ultimate goal should be achieved.
Spectre "Extras"
When Spectre was first implemented, the company was entering into a merger.
It was a requirement that the reporting system could produce output in
the current style as well as a future style (yet to be defined) and that
both styles of reports would be recreatable. So the system had to have
the
capability of producing output in more than one client style. Though
not written with CROs
in mind, this will be of obvious benefit to them since they work for multiple
clients. It also makes Spectre a candidate for becoming a "standard" reporting
engine for use across the industry.
Safety vs. Efficacy Reporting
The Spectre Reporting Macros do both safety and efficacy reporting
in the same report in the way you specify. The reporting macros have powerful
statistical capabilities and huge flexibility to allow you to include efficacy
statistics into the report from whatever source and in any position you
specify. You just have to know the "proc glm" or "proc freq" options to
specify, what ods tables are created as a result and what variables in
those ods tables to use and you can place them whereever you want without
need for further programming. For simple p-values, it is a lot easier.
The User Guides
I hope you found the above introduction to Spectre interesting. Hopefully,
you will have caught a glimpse that this is a reporting engine with purpose
and direction. The next step is to read the User Guides that can be linked
to below. Remember that the Reporting Macros are independent of the
System macros. I have flagged the "system" topics below in brackets
following the page title so that you can skip them if you want to.
You should read the pages in the order listed.
If you are a programmer then you will not need to know about Spectre administration.
The administrator has extra tasks to do. Before you start working on a
study, the administrator will have set up the file "protocol.txt" in your
programs folder and they will have put the file "titles.template" there
which you can copy and use as the template for your titles members. They
will likely be the person to run the suite of programs and to create the
PDFs using the utilities provided. If the suite of programs is large and
the amount of output large then the administrator will have to split the
file "donelist.txt" (the list of all output produced) into "donelist sections"
and to create a PDF from each section. And when the PDFs have been created,
they will be the person to run the "printpdfbookmarks" utility to extract
the list of PDF bookmarks from the PDF files. If Spectre is being used
to create the derived datasets then the administrator will check the naming
convention of the programs to ensure that the suite of programs runs in
the correct order and to ask programmers to rename some of the programs
if this is not the case. They will likely check the naming convention used
for the programs that produce output and sometimes programmers will be
asked to rename their programs (there are utilities to help with this described
on the "Utility Scripts" page). So if you are a programmer, read the administrators
guide if you are interested, but you will find more to interest you in
the "Scripts and Macros" section.
If you are a programmer using Spectre then after a while you will understand
that it is aimed at increasing speed and efficiency. It is there to remove
barriers
to greater productivity, not to add them. Greater speed and efficiency
means that single programmers will maybe end up with a hundred programs
or more. But having a hundred programs or more has the potential of turning
into an administrative nightmare. It would be like winning the technical
battle of efficient coding only to be overwhelmed by sheer weight of numbers.
That would be very stressful for a programmer but help is at hand. Spectre
has many utility scripts to help the programmer manage his hundred or more
programs written using SAS software and it is important for the programmer
to be aware of these scripts and how they can help. Without knowledge of
these scripts, a programmer will eventually hit the barrier caused by sheer
weight of numbers. But knowing that these scripts exist and knowing how
to use them, having hundreds of programs should not be a problem. It will
take almost none of your time and concentration to deal with it.
Here is a typical example of having to divert your attention to work
on one of your hundreds of programs and how you would cope with it.
Suppose someone tells you that they would like the footnote to be changed
for
Table 13.4.5.1 in study XYZ. There is no need to look
up this table in a spreadsheet somewhere. You go to one of your terminal
windows, you probably have variables set up in your ".bashrc" member (sometimes
it is called ".bashrc_own") to help you jump to directories. So you type
in the command "cd $xyzp" (or something like that) to take you to
the study XYZ programs directory. You then type in the command "wt 13.4.5.1"
to tell you what program it is. You then set "prog" equal to this
program and then (assuming "prog" has already been "exported") you type
in the command "titles" and you have the titles member opened in
edit mode. You then change the footnote, recreate the titles members with
the "crtitlesds" command (you may even have an alias for this that
is shorter. I use "crt" as an alias for this). Then when this completes
you type in "sasb". You see that there are no errors or warnings
so you type in "lis" to look at the output. You see that the new
footnote is fine so you close the window and reset the current directory
to what it was before using the command "cd -", and it is done.
You are back working on what you were doing before after only a few seconds
have passed. You hardly noticed the intrusion and now you have even forgotten
about it.
If you know these scripts well and are familiar with Unix commands then
an interruption, such as the one described above, will not break your concentration
and you will find it easy to cope with hundreds of programs. As for writing
programs using SAS software, you will find many macros available to you,
linked to from these pages, to do common difficult tasks. They will help
you to write portable macros that can be used in multiple studies. After
a while, you will find that you are writing less and less code and instead,
copying macros and titles from study to study. There are scripts to help
you do this. After you copy them across, there are scripts to fix the study,
drug and increment names in the headers. There are scripts to help you
do a mass renaming of files. There are scripts to help you tidy up your
".log" and ".lst" files that get left over after a mass renaming has been
done. If you take the trouble to learn about these scripts and macros then
you will easily be able to cope with having hundreds of programs. Hopefully,
you will you become one of the most highly productive clinical reporting
programmers in the field and yet be stress free with it.
The most important macros are %titles, %openrep, %closerep, %popfmt,
%unistats and %npcttab. These have their own pages that you can link to
above and you should make sure you read them thoroughly. The "macros" page
you can link to below just contains summary information about these important
macros. As for the scripts, the ones the programmer will be most interested
in are the "Utility Scripts", as those are the ones that will help speed
and organize their work.
It is important to realise that Spectre is a very simple reporting system.
It is minimilistic and intended that way. It makes available a huge amount
of functionality (particularly with the reporting macros) without expanding
its role and forcing you to change the way you work with it. Instead, it
works for you. Used as a back-end system in the way described here, it
is likely far simpler than any other reporting system currently in use
at any major pharmaceutical company. Many front-end systems become over-complicated,
difficult to use and slow over time, hugely reducing productivity. But
Spectre will always remain as as a powerful, efficient and fast back-end
system. It has a lot of components, as you will see if you look, but then
so does any motorised utility that does something simple like help you
cut your grass. Using Spectre is logical and simple. To help you realise
that, you can link to the page below and see a very simple dummy study,
with just two derived (dummy) datasets and two tables, reported from empty
directories through to final report run, showing all the administrative
tasks performed. Finally, the transformation to get it ready for a "push
buttion" reporting system is demonstrated and explained. If you wanted
to skip through it quickly, it might only take you a few minutes.
I have included three other topics with their own macros that were not
part of the original Spectre as I am sure you will have a use for them
occasionally.
In creating Spectre, I have used some techniques with sas that not all
sas programmers will be familiar with. Without using these techniques,
I could never have written Spectre in the time I did. All the techniques,
as they relate to writing a reporting system, will be explained in the
following document you can link to.
As programmers using Spectre become more proficient, they can find that
they are responsible for hundreds of programs at a time. Dealing with this
many files is easier if you have a good knowledge of Unix commands and
you know how to combine them. This is a higher level of skill than most
sas programmers would have who work on a Unix platform so there are documents
on this web site to help you increase your Unix skills.
If you just want to use the %npcttab and %unistats macros but do not
want to set up the whole of Spectre (Clinical) then installing it is easy.
All you need to do is download the clinmacros and utilmacros
and put them in their own libraries defined to the sasautos path. The rest
of the information you can read here is to do with a full Spectre System
install for those users planning to set up a full clinical reporting system.
Spectre will work best on a Linux or Unix platform with a proper Linux
or Unix version of SAS software (i.e. supporting the -stdio option).
Although it has been ported over to Windows you will have to use the Cygwin
Linux emulator to run the Spectre scripts. On a Windows platform the scripts
to convert output to Postscript files and thence to PDFs will not work
as to get this working would require a lot of disk space and a lot of work
that is probably not needed. If you are a CRO then it is unlikely the client
will ask for your output in PDF format. If they want it in that format
then it is easier if the client accepts your text output files as deliverables
and does the conversion to a PDF themselves. Apart from Postscript files
and PDFs, the rest will work on a Windows platform with Cygwin installed.
It will work fine with SAS Learning Edition 4.1 buth this will limit the
number of observations you can read so you can not use them for production
work.
The full installation of Spectre (Clinical) is quite complex so there
are guides for this that you can link to below.
Spectre System has only a few key components. So long as those components
are not changed, there is huge scope to tailor it so that it works in the
way you want it to. I do not want to give the impression that Spectre System
is a fixed system. It is not. What you see on this web site is more a full-blown
example of what it can do. You can change Spectre System in many ways to
suit, once you have a feel for what these key components are. This is explained
on the following page.
Once you know the key components of Spectre System it should put you in
a position of being able to amend Spectre to suit your site standards.
This is explained on the following page.
The Spectre Clinical reporting macros require a licence for use as of 2012.
The rest is unsupported and free. The licence fee is described on the download
page.
Conclusion
Everything you need to know about Spectre should be documented on this
web site. A few unrelated topics are included in the FAQ which you can
link to below. I hope you will find Spectre fast to learn, easy to use
and that it will increase productivity several fold.
SAS and all other SAS Institute Inc. product or service names are registered
trademarks or trademarks of SAS Institute Inc. in the USA and other countries.
® indicates USA registration.