© Statistisches Bundesamt, I/A
Case study Federal Statistical Office Germany (Destatis)
Joint UNECE/ EUROSTAT/ OECD Work Session on Statistical Metadata (METIS) Lisbon, 11 – 13 March, 2009
© Statistisches Bundesamt, I/A
About lessons learned
About process models
About the systems
About usI. Who we are
II. Where we started
III. What is already there
IV. What is being implemented
V. What is planned for the future
VI. How we use process models
VII.What we have learned
Content
© Statistisches Bundesamt, I/A
Who we are
Metadata management unit – 2-3 people
Established in mid-2007
Part of central coordination department
Several working groups and projects
© Statistisches Bundesamt, I/A
DESTATIS
Central authority in official statistics in Germany
2710 employees (as of July 2008)
50.5% with graduate degree (16.4% with upper post-secondary degree)
© Statistisches Bundesamt, I/A
Official Statistics in Germany
14 Länder offices – 1 Federal Statistical Office (Destatis)
Organizationally entirely independent offices
Many statistical activities are carried out jointly by federal and Länder offices
© Statistisches Bundesamt, I/A
Work sharing in the statistical system
Federal Statistical Office (Destatis) organizes federal statistical activities …
… and is responsible for the methodology
Länder offices carry out the core statistical processes
© Statistisches Bundesamt, I/A
Joint decision making in the statistical system
Several decision making bodies and working groups on different levels
… applies also to IT
Statistical offices in Germany share existing and develop new IT-systems together
© Statistisches Bundesamt, I/A
Where we started
Some systems in place, when we started
New project to standardize business procedures and IT-systems: “SteP – Standardisierung der Produktion”
No common metadata strategy – no agreed metadata model
Several requirements by governing committees
Metadata management for Census 2011
© Statistisches Bundesamt, I/A
GENESIS-Online
RDC Metadata-system
Survey Database
Statistical activitiesdatabase
QualityreportsIn-house
systemsSAS
Excel
???
Collect Process Analyze Disseminate Design
Internet
data editing tool
What is already there
edr tool
electronic questionnaire tool
© Statistisches Bundesamt, I/A
What is already there GENESIS-database
output database, cube based, extensive data and metadata model, stores metadata internally (about 1,300 cube-variables), supports sdmx-queries
RDC-metadata systemtechnically a spin-off from GENESIS to store metadata that describes micro data files for researchers, successfully implemented, great level of detail (currently 5,600 variables used in data files for 33 statistical activities)
© Statistisches Bundesamt, I/A
What is already there .BASE
in-house production system to support an end-to-end workflow, consists of a central database for metadata (in the form of XML-files), current tools that run on this metadata: data editing tool, edr-tool, electronic questionnaire tool, no (standard) conceptual model, metadata on a rather technical level
© Statistisches Bundesamt, I/A
What is being implemented Klass-Service
Neuchâtel-based classification server, carried out in collaboration with Destatis’ classification unit, in fact a (very thorough) reengineering of an older system, collaboration with Statistics Norway, implementation: Bavarian State Office for Statistics and Data Processing
Reengineering of Statistikdatenbankinternal information system, registers all statistical activities (on a higher level)
© Statistisches Bundesamt, I/A
What is planned Metadata systems for the Census 2011,
among them:
Variable serveras part of the census project, designed after consultation with the Swiss Statistical Office, ought to be Neuchâtel-based as well, would ideally provide a connection between output systems (e.g. GENESIS) and processing systems (e.g. .BASE)
© Statistisches Bundesamt, I/A
What is planned Also upcoming:
Organizational matters
Organizational implementation of the systems
Drafting of a strategy
Making conceptual model and process model known in other parts of the organization
© Statistisches Bundesamt, I/A
How we use process models
Management processes
Support processes
Collect Process AnalyzeCore
processes
Development Production Interpretation
Manage statistical activities
Develop methodology
Design Disseminate
Supply infrastructure
Supply resources
Manage human
resourcesOrganize
Communicate internally
Manage operationsManage Marketing and
Public RelationsManage people
Devise strategy
Desta
tis P
rocess M
od
el
© Statistisches Bundesamt, I/A
How we use process models
1. Print questionnaire
3. Monitor responses and follow-up non-respondents
2. Send questionnaire
4. Manage interviewers
5. Import data
6. Update registers and other reference files
7. Draw sample
8. Initial Check
9. Code and classify responses
10. Derive new variables
11. Perform micro editing
12. Calculation
13a. Estimation + 13b. Estimate sampling error
14. Check data protection
15. Finalise output tables
16. Exchange output data
17. Archive (micro) data
18. Analyse micro data further
19. Promote products
20. Manage respondents
SteP Process Model
3.Survey
Preparation
4.Survey Execution
Specify NeedsBuild
2Design
1Disseminate
6Analyse
5
Define business processes
0.5
Establish objectives
0.3
Check data availability
0.4
Consult and confirm
0.2
Determine need for information
0.1
Load data into electronic (standard)
format4.2
Run data collection
4.1
Calculate weights4.7
Impute missing data4.5
Generate new variables and
statistical units4.6
Finalise data files4.9
Integrate data4.8
Edit data4.4
Classify and code4.3
Prepare statistics for dissemination
5.4
Check quality of statistics
5,2
Interpret and explain statistics
5.3
Finalise statistics5.5
Specify and produce statistics
5.1
Promote products6.5
Release products6.3
Manage customers
6.4
Produce products6.2
Manage output database
6.1
Design Output1.1
Design IT-requirements
1.6
Design methodology
1.3
Define populations and variables
1.2
Configure workflow
3.1
Finalise IT-system for production
2.4
Run field test2.5
Test IT-systems2.3
Define survey processes
1.5
Produce new / (Re-)Configure old
IT-systems2.2
Configure and test collection
instruments 2.6
Archive 7
Dispose of data and metadata
7.3
Archive data and metadata
7.2
Manage archive7.1
Collect & Process3 & 4
Prepare data collection
3.4
Manage data providers
3.3
Draw sample / Produce lists
3.2
Draft laws1.4
Evaluate 8
Execute evaluation survey
8.1
Design IT-systems2.1
Validate metadata
Validate metadata
Validate metadata
Validate metadata
1.Business Case Development
2.IT-
Development
5.Analysis
Census 2011 Processes
Sub
-Pro
cess
Process
Disseminate6
Analyse5
Load data into electronic (standard) format
4.2
Run data collection4.1
Calculate weights4.7
Impute missing data4.5
Generate new variables and statistical units
4.6
Finalise data files4.9
Integrate data4.8
Edit data4.4
Classify and code4.3
Prepare statistics for dissemination
5.4
Check quality of statistics
5,2
Interpret and explain statistics
5.3
Finalise statistics5.5
Specify and produce statistics
5.1
Promote products6.5
Release products6.3
Manage customers6.4
Produce products6.2
Manage output database
6.1
Configure workflow 3.1
Archive 7
Dispose of data and metadata
7.3
Archive data and metadata
7.2
Manage archive7.1
Collect & Process3 & 4
Prepare data collection
3.4
Manage data providers
3.3
Draw sample / Produce lists
3.2
Sub
-Pro
cess
Process
3.4.1 = 1. Print questionnaires (applies to all surveys with collection
instrument = paper questionnaire)
3.4.2 = 2. Send questionnaires (applies to all surveys with collection instrument = self administered paper
questionnaire)
4.1.1 = 3. Monitor responses and follow-up non-respondents
4.1.1 = 5. Import data (applies to all surveys with collection instrument =
electronic data transmission
Not included = 4. Manage interviewers
3.3.1 = 6. Update registers and other reference files
3.2.1 = 7. Draw sample (applies to all surveys with sampling method =
probability sampling)
4.1.1 = 8. Initial check (there are no details on this process)
4.3 = 9. Classify and code
4.4 = 11. Edit data
4.5 = 10. Generate new variables and statistical units
Not included = 12. Calculations
= 13a. Projection (applies to all surveys with sampling method =
probability sampling)
5.2 = 13b. Estimate sampling error (applies to all surveys with sampling
method = probability sampling)
5.4 = 14. Check data protection (can also be part of 4. Process depending on method of data
protection)
5.5 = 15. Finalise output tables
Not included = 16. Exchange output data
7.2 = 17. Archive (micro) data
Not included = 18. Analyse micro data further
6.5 = 19. Promote products
6.5 = 20. Manage respondents
© Statistisches Bundesamt, I/A
What we have learned
Communication is important
Strategy is needed for coordination
Learn from previous implementations
Use existing models like Neuchâtel/ISO 11179
Cooperate internationally
© Statistisches Bundesamt, I/A
Thank You For Your Attention!