\chapter{Panel data}
\label{chap-panel}

\section{Panel structure}
\label{panel-structure}

Panel data are inherently three dimensional --- the dimensions being
variable, cross-sectional unit, and time-period.  For representation
in a textual computer file (and also for gretl's internal
calculations) these three dimensions must somehow be flattened into
two.  This ``flattening'' involves taking layers of the data that
would naturally stack in a third dimension, and stacking them in the
vertical dimension.

\app{Gretl} always expects data to be arranged ``by observation'',
that is, such that each row represents an observation (and each
variable occupies one and only one column).  In this context the
flattening of a panel data set can be done in either of two ways:

\begin{itemize}
\item Stacked cross-sections: the successive vertical blocks each
  comprise a cross-section for a given period.
\item Stacked time-series: the successive vertical blocks each
  comprise a time series for a given cross-sectional unit.
\end{itemize}

You may use whichever arrangement is more convenient. Under
\app{gretl}'s \textsf{Sample} menu you will find an item ``Restructure
panel'' which allows you to convert from stacked cross section form to
stacked time series or vice versa.

When you import panel data into \app{gretl} from a spreadsheet or
comma separated format, the panel nature of the data will not be
recognized automatically (most likely the data will be treated as
``undated'').  A panel interpretation can be imposed on the data in
either of two ways.

\begin{enumerate}
\item Use the GUI menu item ``Sample, Dataset structure''.  In the
  first dialog box that appears, select ``Panel''.  In the next
  dialog, make a selection between stacked time series or stacked
  cross sections depending on how your data are organized.  In the
  next, supply the number of cross-sectional units in the dataset.
  Finally, check the specification that is shown to you, and confirm
  the change if it looks OK.
\item Use the script command \cmd{setobs}.  For panel data this
  command takes the form \verb+setobs+ \textsl{freq} \verb+1:1+
  structure, where \textsl{freq} is replaced by the ``block size'' of
  the data (that is, the number of periods in the case of stacked time
  series, or the number of cross-sectional units in the case of
  stacked cross-sections) and structure is either
  \verb+--stacked-time-series+ or \verb+--stacked-cross-section+.  Two
  examples are given below: the first is suitable for a panel in the
  form of stacked time series with observations from 20 periods; the
  second for stacked cross sections with 5 cross-sectional units.
\begin{code}
            setobs 20 1:1 --stacked-time-series
            setobs 5 1:1 --stacked-cross-section
\end{code}
\end{enumerate}

\subsection{Panel data arranged by variable}

Publicly available panel data sometimes come arranged ``by variable.''
Suppose we have data on two variables, \varname{x1} and \varname{x2},
for each of 50 states in each of 5 years (giving a total of 250
observations per variable).  One textual representation of such a data
set would start with a block for \varname{x1}, with 50 rows
corresponding to the states and 5 columns corresponding to the years.
This would be followed, vertically, by a block with the same structure
for variable \varname{x2}.  A fragment of such a data file is shown
below, with quinquennial observations 1965--1985.  Imagine the table
continued for 48 more states, followed by another 50 rows for variable
\varname{x2}.

\begin{center}
  \begin{tabular}{rrrrrr}
  \varname{x1} \\
     & 1965 & 1970 & 1975 & 1980 & 1985 \\
  AR & 100.0 & 110.5 & 118.7 & 131.2 & 160.4\\
  AZ & 100.0 & 104.3 & 113.8 & 120.9 & 140.6\\
  \end{tabular}
\end{center}

If a datafile with this sort of structure is read into \app{gretl},
the program will interpret the columns as distinct variables, so the
data will not be usable ``as is.''  But there is a mechanism for
correcting the situation, namely the \cmd{stack} function within
the \cmd{genr} command.

Consider the first data column in the fragment above: the first 50 rows
of this column constitute a cross-section for the variable \varname{x1}
in the year 1965.  If we could create a new variable by stacking the
first 50 entries in the second column underneath the first 50 entries
in the first, we would be on the way to making a data set ``by
observation'' (in the first of the two forms mentioned above, stacked
cross-sections).  That is, we'd have a column comprising a
cross-section for \varname{x1} in 1965, followed by a cross-section for
the same variable in 1970.

The following gretl script illustrates how we can accomplish the
stacking, for both \varname{x1} and \varname{x2}.  We assume
that the original data file is called \texttt{panel.txt}, and that in
this file the columns are headed with ``variable names'' \varname{p1},
\varname{p2}, \dots, \varname{p5}.  (The columns are not really
variables, but in the first instance we ``pretend'' that they are.)

\begin{code}
    open panel.txt
    genr x1 = stack(p1..p5) --length=50
    genr x2 = stack(p1..p5) --offset=50 --length=50
    setobs 50 1.01 --stacked-cross-section
    store panel.gdt x1 x2
\end{code}

The second line illustrates the syntax of the \cmd{stack} function.
The double dots within the parentheses indicate a range of variables
to be stacked: here we want to stack all 5 columns (for all 5 years).
The full data set contains 100 rows; in the stacking of variable
\varname{x1} we wish to read only the first 50 rows from each column:
we achieve this by adding \verb+--length=50+.  Note that if you want
to stack a non-contiguous set of columns you can put a comma-separated
list within the parentheses, as in

\begin{code}
    genr x = stack(p1,p3,p5)
\end{code}

On line 3 we do the stacking for variable \varname{x2}.  Again we want
a \texttt{length} of 50 for the components of the stacked series, but
this time we want gretl to start reading from the 50th row of the
original data, and we specify \verb+--offset=50+.

Line 4 imposes a panel interpretation on the data, as explained in
section~\ref{panel-structure}.  Finally, we save the data in gretl
format, with the panel interpretation, discarding the original
``variables'' \varname{p1} through \varname{p5}.

The illustrative script above is appropriate when the number of
variable to be processed is small.  When then are many variables in
the data set it will be more efficient to use a command loop to
accomplish the stacking, as shown in the following script.  The setup
is presumed to be the same as in the previous section (50 units, 5
periods), but with 20 variables rather than 2.

\begin{code}
    open panel.txt
    loop for i=1..20
      genr k = ($i - 1) * 50
      genr x$i = stack(p1..p5) --offset=k --length=50
    endloop
    setobs 50 1.01 --stacked-cross-section
    store panel.gdt x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 \
      x11 x12 x13 x14 x15 x16 x17 x18 x19 x20
\end{code}


\section{Dummy variables}
\label{dummies}

In a panel study you may wish to construct dummy variables of one or
both of the following sorts: (a) dummies as unique identifiers for the
cross-sectional units, and (b) dummies as unique identifiers for the
time periods.  The former may be used to allow the intercept of the
regression to differ across the units, the latter to allow the
intercept to differ across periods.

Three special functions are available to create such dummies.  These
are found under the ``Data, Add variables'' menu in the GUI, or under
the \cmd{genr} command in script mode or \app{gretlcli}.

\begin{enumerate}
\item ``periodic dummies'' (script command \cmd{genr dummy}).  This
  command creates a set of dummy variables identifying the periods.
  The variable \verb+dummy_1+ will have value 1 in each row
  corresponding to a period 1 observation, 0 otherwise; \verb+dummy_2+
  will have value 1 in each row corresponding to a period 2
  observation, 0 otherwise; and so on.
\item ``unit dummies'' (script command \cmd{genr unitdum}).  This
  command creates a set of dummy variables identifying the
  cross-sectional units.  The variable \verb+du_1+ will have value 1
  in each row corresponding to a unit 1 observation, 0 otherwise;
  \verb+du_2+ will have value 1 in each row corresponding to a unit 2
  observation, 0 otherwise; and so on.
\item ``panel dummies'' (script command \cmd{genr paneldum}).  This
  creates both period and unit dummy variables. The unit dummies are
  named \verb+du_1+, \verb+du_2+ and so on, while the period dummies
  are named \verb+dt_1+, \verb+dt_2+, etc.
\end{enumerate}

If a panel data set has the \verb+YEAR+ of the observation entered as
one of the variables you can create a periodic dummy to pick out a
particular year, e.g.\ \cmd{genr dum = (YEAR=1960)}.  You can also
create periodic dummy variables using the modulus operator,
\verb+%+.  For instance, to create a dummy with
value 1 for the first observation and every thirtieth observation
thereafter, 0 otherwise, do
\begin{code}
      genr index 
      genr dum = ((index-1)%30) = 0
\end{code}

\section{Lags and differences with panel data}
\label{panel-lagged}

If the time periods are evenly spaced you may want to use lagged
values of variables in a panel regression; you may also with to
construct first differences of variables of interest.

Once a dataset is properly identified as a panel, \app{gretl} will
handle the generation of such variables correctly.  For example the
command \verb+genr x1_1 = x1(-1)+ will create a variable that contains
the first lag of \verb+x1+ where available, and the missing value code
where the lag is not available.  When you run a regression using such
variables, the program will automatically skip the missing
observations.

\section{Pooled estimation}
\label{pooled-est}

There is a special purpose estimation command for use with panel data,
the ``Pooled OLS'' option under the \textsf{Model} menu. This command
is available only if the data set is recognized as a panel.  To take
advantage of it, you should specify a model without any dummy
variables representing cross-sectional units.  The routine presents
estimates for straightforward pooled OLS, which treats cross-sectional
and time-series variation at par.  This model may or may not be
appropriate.  Under the \textsf{Tests} menu in the model window, you
will find an item ``panel diagnostics'', which tests pooled OLS
against the principal alternatives, the fixed effects and random
effects models.

The \emph{fixed effects} model adds a dummy variable for all but one
of the cross-sectional units, allowing the intercept of the regression
to vary across the units.  An \emph{F}-test for the joint significance
of these dummies is presented: if the p-value for this test is small,
that counts against the null hypothesis (that the simple pooled model
is adequate) and in favor of the fixed effects model.

The \emph{random effects} model, on the other hand, decomposes the
residual variance into two parts, one part specific to the
cross-sectional unit or ``group'' and the other specific to the
particular observation.  (This estimator can be computed only if the
panel is ``wide'' enough, that is, if the number of cross-sectional
units in the data set exceeds the number of parameters to be
estimated.)  The Breusch--Pagan LM statistic tests the null hypothesis
(again, that the pooled OLS estimator is adequate) against the random
effects alternative.

It is quite possible that the pooled OLS model is rejected against
both of the alternatives, fixed effects and random effects. How, then,
do we assess the relative merits of the two alternative estimators?
The Hausman test (also reported, provided the random effects model can
be estimated) addresses this issue.  If the unit- or group-specific
error is uncorrelated with the independent variables, the random
effects estimator is more efficient than the fixed effects estimator;
otherwise the random effects estimator is inconsistent, in which case
the fixed effects estimator is to be preferred.  The null hypothesis
for the Hausman test is that the group-specific error is not so
correlated (and therefore the random effects model is preferable).
Thus a low p-value for this tests counts against the random effects
model and in favor of fixed effects.

For a rigorous discussion of this topic, see Greene (2000), chapter
14.

\section{Illustration: the Penn World Table}
\label{PWT}

The Penn World Table (homepage at
\href{http://pwt.econ.upenn.edu/}{pwt.econ.upenn.edu}) is a rich
macroeconomic panel dataset, spanning 152 countries over the years
1950--1992.  The data are available in \app{gretl} format; please see
the \app{gretl}
\href{http://gretl.sourceforge.net/gretl_data.html}{data site} (this
is a free download, although it is not included in the main
\app{gretl} package).

Example \ref{examp-pwt} opens \verb+pwt56_60_89.gdt+, a subset
of the PWT containing data on 120 countries, 1960--89, for 20
variables, with no missing observations (the full data set, which is
also supplied in the pwt package for \app{gretl}, has many missing
observations). Total growth of real GDP, 1960--89, is calculated for
each country and regressed against the 1960 level of real GDP, to see
if there is evidence for ``convergence'' (i.e.\ faster growth on the
part of countries starting from a low base).

\begin{script}[htbp]
  \caption{Use of the Penn World Table}
  \label{examp-pwt}
\begin{code}
          open pwt56_60_89.gdt 
          # for 1989 (the last obs), lag 29 gives 1960, the first obs 
          genr gdp60 = RGDPL(-29) 
          # find total growth of real GDP over 30 years
          genr gdpgro = (RGDPL - gdp60)/gdp60
          # restrict the sample to a 1989 cross-section 
          smpl --restrict YEAR=1989 
          # convergence: did countries with a lower base grow faster?  
          ols gdpgro const gdp60 
          # result: No! Try an inverse relationship?
          genr gdp60inv = 1/gdp60 
          ols gdpgro const gdp60inv 
          # no again.  Try treating Africa as special? 
          genr afdum = (CCODE = 1)
          genr afslope = afdum * gdp60 
          ols gdpgro const afdum gdp60 afslope 
\end{code}
\end{script}


%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "gretl-guide"
%%% End: 

