Zakir Hussain  ( Department of Health, Gilgit-Baltistan, Pakistan )
Jameel Ahmed Ansari  ( Field epidemiology and disease surveillance division National institute of Health Islamabad, Pakistan. )
Muhammad Salman  ( Pathologist, National institute of Health Islamabad, Pakistan. )
Ejaz Ahmed Khan  ( Director Surveillance, Expanded program on Immunization Islamabad, Pakistan. )
Jawad Asghar  ( Resident Advisor, Field epidemiology and Laboratory training program Pakistan, DC Atlanta USA. FELTP, N.I.H Islamabad Pakistan. )

Background: Acute Respiratory Infections (ARIs) are leading cause of death among children and cause 2.1 million deaths worldwide. In Pakistan, around 11.74 million ARIs cases are reported annually. Gilgit Baltistan (G-B) has higher ARIs incidence and mortality among children under 5. Study aims to evaluate the strengths and weaknesses of ARI surveillance systems in GB.

Methodology: Evaluation was conducted during April 2011 using CDC\\\'s updated guidelines on evaluating the public health surveillance system, 2001. Two existing surveillance systems: Health Management Information System (HMIS) and Lady Health Workers Management Information System (LHWs-MIS) were identified. Literature, documents and reports were reviewed. Stake holders were interviewed. Sensitivity was calculated using estimates of Pakistan Health & Demographic Survey (PDHS) 2006-07.
Results: HMIS was simple, but inflexible in accommodating additional information. Data quality was poor due to incomplete forms; data took 30 days to reach provincial HMIS cell. Sensitivity of system was 94% but lacked Predictive Value Positive (PVP) due to absence of laboratory confirmation. System was operational in public sector and covered 95% population in 95% districts. System was stable with timely and secure data collection and management. LHWs-MIS, was simple and lacked flexibility. Data quality was good and stable with adequate collection, management and provision of data. Form completion was around 90%. However, had moderate representativeness covering about 65% districts. Data were transmitted within 30 days while the Sensitivity was 84%.
Conclusion: LHW/MIS has better quality data and timeliness, while HMIS has better coverage. Both systems have different objectives and have problems in flexibility along with an absence of lab component. To cater for the robustness of the surveillance system, new information system with latest and appropriate technology is required.
Keywords: ARI, Evaluation, Surveillance System, Gilgit-Baltistan. (JPMA 66: 682; 2016)

Introduction
Acute Respiratory Infections (ARIs) are classified as upper respiratory tract infections (URTI) or lower respiratory tract infections (LRTI). URTIs are usually not life threatening. Whereas the LRTIs such as Pneumonia and Bronchitis/bronchiolitis are associated with significant morbidity and mortality in developing countries.^1,2 WHO has provided a standard protocol for the diagnosis and assessment of ARIs (case definitions) which is based on the presence of cough, tachypnoea, chest in drawing, and wheezing for 7 days. Severe disease is defined in children with a respiratory rate >60/ minute and chest in-drawing, lethargy and inability to drink.³ Globally 2.1 million deaths among under five years age group are attributed to ARIs, accounting for about 20% of all childhood deaths;3 primarily in developing countries.⁴ Poverty, low family income, parental illiteracy, high proportion of low birth weight babies, malnutrition, and ignorance of exclusive breastfeeding are considered major risk factors for development of respiratory infections.⁵
In Pakistan, ARIs are responsible for 20-30% of all deaths among children under age of 5 years.⁶ Pakistan demographic and health survey 2006-2007 shows that fourteen percent (14%) of children less than five years (< 5) age have had symptoms of ARI.⁷ For the year 2006, incidence of the ARI was 2165 cases per ten thousand population and in year 2007 it declined to 1850. Annually around 11.74 million cases of ARIs are reported in different health facilities of Pakistan.⁸
ARIs incidence rates in province Gilgit-Baltistan are much higher than the rates reported at lower altitudes in the country,9 80% of children less than five years presented to the health facilities with acute ARIs.10,11 For the treatment of an outpatient case of childhood pneumonia, the average cost of treatment is around 13.44 dollars, and for indoor admissions, the public sector health system has been spending an average of US$ 71 per admission for mild pneumonia and US$ 235 for severe pneumonia.¹²
Public health surveillance has been defined as "an ongoing, systematic collection, analysis, interpretation and dissemination of data linked with public health action to reduce morbidity and mortality and to improve health populations".^13,14 Surveillance helps in detecting disease, support health organizations and interventions, estimating the impact of a disease or injury, portraying the natural history of diseases.^14,15 Evaluation of a public health Surveillance system gives a clear description of purpose, design, management and operational characteristics of the system and provides evidence based information which could be used for strengthening the disease reporting mechanism and enhancing implementation of public health action.¹⁶
Under the International Health Regulations (IHR) 2007, the World Health Organization (WHO) member states are required to strengthen their surveillance systems by using existing national infrastructures and resources.^17-20
HMIS was developed in Pakistan in the early 1990s by defunct Ministry of Health (MoH) Islamabad. This system aimed to provide information on facility utilization rate, referrals, immunization, maternal care and family planning, provides information on 18 priority diseases and control of those diseases through regular and timely reporting with monitoring of the drugs availability, contraceptives, functionality of equipment, repair and maintenance of facilities and utilities. Information generated at facility level is transmitted to national HMIS cell through district and provincial setups (Figure-1)


Lady Health Workers-Management Information System (LHWs-MIS) is a community based information system developed in 1994 by the National Programme for Family Planning and Primary Health Care (FP & PHC) selected health facilities were designated as First level care facilities (FLCFs). LHWs are submitting reports on monthly basis at health facilities from where these reports are sent to district and provincial offices21 (Figure-2)

.
The objectives of LHWs-MIS is to collect information regarding mother and child health in community, provide a source of verification of LHWs activities for health facility staff and programme monitors by linking the information collected by LHWs with basic health facilities information.^21-23 This evaluation therefore aimed to assess strengths and weaknesses of existing reporting system(s) for ARIs in Gilgit-Baltistan (GB), identify gaps and suggest measures to improve surveillance system.24

Methodology
This evaluation was carried out in March 2011 at Directorate of Health, Department of Health Gilgit-Baltistan. We evaluated the ARI reporting surveillance system in the GB by use of the Centre for disease control and prevention (CDC) USA updated guidelines for evaluation of Public Health Surveillance.²⁵ We collected required information on both systems through review of documents, records, registers, published literature and in-person interviews. Qualitative analysis provided information about system attributes; simplicity, flexibility, data quality, acceptability, timeliness, representativeness and stability while quantitative analysis provided the sensitivity and Positive Predictive Value (PVP) of the systems. Based on the information collected, we describe standard system attributes for evaluating the public health surveillance.

Results
Based on the information; two surveillance systems were identified: (a) Health Management Information System (HMIS), and (b) Leady Health Workers\\\'-medical information system (LHWs-MIS).
1. Health Management information system (HMIS)
Attributes of the systems
A summary of the evaluation of the 09 attributes from the CDC guidelines are presented in Table.
Simplicity: The HMIS was found to be simple to operate being manually reported on user-friendly hand-written forms. Case definition has been met, no Integration with other systems.
Staff not well trained in data management and transferring.
Flexibility: The system was not flexible as it was found being overloaded and having limited capacity to accommodate any additional information on new disease or health event. The system did not give any information about respiratory infection types and confirmation of cases. (Table-1)


Data Quality: There was no data quality check system. Trained staff for data compilation and analysis was not available at the facility and district level both.
Acceptability: All health managers and HMIS staff at all levels were willing to work and were reporting on regular basis. Provincial health departments have had established respective provincial HMIS cells. Nonetheless, the required operational expenses for the purpose were not provided on regular basis.
Sensitivity: The Prevalence of ARI was found to be 14% (11). It clearly showed an estimated number of ARI cases should have had been 38,080 cases/ years while the ARI cases detected by system were34, 169.Sensitivity of system to detect ARI cases, was therefore 94 %.
Predictive Value Positive: Could not be estimated as syndromic case definition was used with no laboratory component being involved
Representativeness: HMIS covered primary health care facilities of seven districts in the GB. On an average, monthly reporting rate ranged from 86% in 2009 to 91% in 2010 (an increase of 5%). Information from DHQs hospitals and private sector was not being fed to the HMIS. Therefore HMIS was not a truly representative disease surveillance system for ARIs.
Timeliness: Reporting Timeliness of information by districts remained 83% (n=05) as 05 districts out of 06 submitted reports timely in 2008 and 66% (n=04) in 2009, HMIS data takes 1-2 weeks to reach district HMIS cell and >25 days to reach Provincial HMIS cell. (Figure-3)

. The District HMIS cell transmitting information to the provincial HMIS cell was without any feedback to the first level care health facilities (FLCFs).
Stability: The provincial cell also did not provide regular feedback to districts, and transmitted data to national HMIS cell using DOS-based software through internet. Computers, internet and software\\\'s were not available at health facilities and District HMIS cell level. All type of reporting from the FLCFs to the districts was in the form of paper-based reports.
From the provincial to the federal level, the information was sent through internet using DOS based software. Due to manual form used, of there is always a chance of information misplacement at all levels.
2. Lady Health workers Management information system (LHWs-MIS)
Simplicity: LHWs-MIS Reporting forms were simple to fill and LHWs and other MIS related staff were regularly being trained to fill registers and monthly report forms.
Flexibility: System was also not flexible to accommodate new health related events. Staff was designated and working at all levels for LHWs MIS.
Data Quality: Data quality in terms of reporting, compilation and analysis and feedback was good. Data quality check system practically existed through regular monitoring by district coordinators and field programme Officers. The system has mechanisms to check for errors. LHWs followed WHO Guidelines for the identification and reporting of the ARI cases. Syndromic case definition was being used to report cases. Provincial Implementation Unit (PIU) regularly provided the operational expenses and printed material to district and other facilities.
Acceptability: All managers, supervisors are designated and willing to work at all levels, interview completion rate was 100% without any refusal
Sensitivity: The system detected on average 28,792 cases of ARI each year. Prevalence of ARI was 14% and the system should have had picked 38,080 ARI cases each year thus the sensitivity of information system to detect ARI cases was 81%.
Predictive Value Positive (PVP): Due to Syndromic case definition used to identify cases, and without any laboratory confirmation or gold standard test; it was hard to measure Positive Predictive Value (PPV or PVP).
Representativeness: This system covered all seven districts but did not cover all health facilities across GB and DHQ hospitals were not covered as well. It provided information only from the LHWs covered areas while LHWs coverage in the GB was only 65%.
Timeliness: Out of 96 FLCFs in six districts, 88 FLCFs submitted reports on time during last two years n=88 (91 %). Reports took 25 days to reach federal unit from first level care facility, whereas the data analysis and feedback took longer time then required for response to an outbreak (Figure-4)

.
Stability: System is stable in terms of continuity of functions, Staff is designated and working at all levels System has ability to collect ,manage and provide data whenever needed without failure The cost involved for repair ,maintenance and operation is provided regularly.

Discussion
HMIS was simple, but inflexible in accommodating additional information. Data quality was poor due to incomplete forms; data took 30 days to reach provincial HMIS cell. Sensitivity of system was 94% but lacked Predictive Value due to absence of laboratory confirmation system. Although syndromic surveillance is capable of detecting ARIs outbreaks timely and are not dependent on Lab-based outbreak detection, we argue that lack of PVP leads to non-compliance to the objective of the system.^15,26 The system also lacked coverage of about 80% of private healthcare, which is also true for rest of the country. Surveillance of the infectious diseases can be improved in the country through developing a system which also takes the private sector into account.^27-30 LHWs-MIS, was simple but again lacked flexibility. Apart from some optimal performance, it had moderate representativeness covering around two-thirds of the districts. Data transmission took about a month with good sensitivity. The delayed data transmission, lack of analysis and response mechanisms compromises the information system. We compared three years reports of both systems for timeliness of reporting which was found to be not sufficient to fulfil objectives of surveillance systems. Poor timeliness of surveillance system reflects the delay in timely decision of the health managers and policy makers. Findings from a review of published papers in United States in 2004 also concludes that the long reporting lag and the variability, limit the usefulness of surveillance data and timely response to outbreaks and recommends a more standardized methods may be used for measurement of timeliness of a public health surveillance system which should be in detail, clear and without any confusion or doubt.²¹
HMIS is however; limited to first level care facilities as major District hospitals (DHQs) are not reported in system therefore the system does not give a very true picture of the occurrence of ARI. The system is not able to detect and respond to outbreaks due to delayed reporting and limited data use at local and the provincial level. Use of latest technology, new android-based cellular phones and GPS tracking systems should improve the surveillance system in Pakistan.³¹

Conclusion
There is no integration and sharing of information between these two information systems, therefore there are chances of duplication of reported cases and feedback mechanism does not exists at all levels. LHW/MIS has better quality data and timeliness, while HMIS has better coverage. Both systems have different objectives and have problems in flexibility along with an absence of lab component. To cater for the robustness of the surveillance system, new information system with latest and appropriate technology is required.

Acknowledgements
Field Epidemiology and Laboratory Program (FELTP) Pakistan Islamabad have supported this descriptive study. Directorate Health Gilgit-Baltistan, Provincial Coordinator National program for family planning and primary Health care (LHWs Program) Regional Directorate of Health Services Baltistan supported in provision of access to necessary documents/information for this evaluation.

Disclosure: The article or part of the article has not been published in any other journal.

Conflict of Interest: None.

Financial Support: None.

References
1. Simoes EAF, Cherian T, Chow J, Shahid-Salles SA, Laxminarayan R, John TJ. Acute Respiratory Infections in Children. In: Jamison DT, Breman JG, Measham AR, Alleyne G, Claeson M, Evans DB, Jha P, Mills A, Musgrove P, editors. Disease Control Priorities in Developing Countries. 2nd edition. Washington (DC): World Bank; 2006. Chapter 25.
2. Simoes EA. Environmental and demographic risk factors for respiratory syncytial virus lower respiratory tract disease. J Pediatrics. 2003; 143:118-26.
3. Murray CJ, Lopez AD, Mathers CD, Stein C. The Global Burden of Disease 2000 project: aims, methods and data sources. Geneva: World Health Organization, 2001.
4. Bryce J, Boschi-Pinto C, Shibuya K, Black RE. WHO estimates of the causes of death in children. Lancet. 2005; 365:1147-52.
5. Cashat-Cruz M, Morales-Aguirre JJ, Mendoza-Azpiri M. Respiratory tract infections in children in developing countries. Semin Pediatr Infect Dis. 2005;16:84-92.
6. Khan TA, SA Madni, Zaidi A. Acute respiratory infections in Pakistan: have we made any progress? J Coll Phyisicians Surg Pak. 2004; 14:440-8.
7. Demographic P. Health Survey 2006-2007 [Preliminary report]. National Institute of Population Studies. Islamabad, Pakistan. Columbia MA: IRD/Macro International, 2007; p 25.
8. Ali M, Horikoshi Y. Situation analysis of health management information system in Pakistan. Pakistan J Med Res. 2002; 41:64-9.
9. Khan AJ, Hamidah H, Saad B, Sajida C, Sajid A, Adil K. et al. High incidence of childhood pneumonia at high altitudes in Pakistan: a longitudinal cohort study. Bull World Health Organ. 2009; 87:193-9.
10.cVarley E. Targeted doctors, missing patients: Obstetric health services and sectarian conflict in Northern Pakistan. Social Sci Medicine. 2010; 70:61-70.
11.Anwar M, Green J, Norris P. Health-seeking behaviour in Pakistan: A narrative review of the existing literature. Public Health. 2012; 126: 507-17.
12. Hussain H, Waters H, Omer SB, Khan A, Baig IY, Mistry R. etal. The cost of treatment for child pneumonias and meningitis in the Northern Areas of Pakistan. Int J Health Planning Manag. 2006; 21: 229-38.
13. Pio A, Standard case management of pneumonia in children in developing countries: the cornerstone of the acute respiratory infection programme. Bull World Health Organ 2003; 81:298-300.
14. Wharton M, Terence L, Richard L, Dale L, James W, Chorba. et al. Case definitions for public health surveillance. MMWR Recomm Rep. 1990; 39:1-43.
15. van den Wijngaard CC. Is This an Outbreak? A retrospective evaluation of syndromic surveillance for emerging infectious disease. Erasmus MC: University Medical Center Rotterdam, 2010.
16. Gazarian M, Katrina W, Elizabeth E, Kerry C, Helen L,Craig M. et al. Evaluation of a national surveillance unit. Arch Dis Childhood. 1999; 80:21-7.
17. Baker MG, DP Fidler. Global public health surveillance under new international health regulations. Emerging infectious diseases. 2006; 12:1058.
18. Organization WHO, International Health Regulations (2005). World Health Organization, 2008.
19. Fidler DP, LO Gostin, The new International Health Regulations: an historic development for international law and public health. J Law, Medicine Ethics. 2006; 34:85-94.
20. Wilson K, B von Tigerstrom, C McDougall. Protecting global health security through the International Health Regulations: requirements and challenges. Canadian Med Assoc J. 2008; 179: 44-8.
21. Ali M, Horikoshi Y. Situation analysis of health management information system in Pakistan. Pakistan J Med Res. 2002; 41: 64-9.
22. Sherwani J, Ali N, Mirza S, Fatma A, Memon Y, Karim M. et al. Healthline: Speech-based access to health information by low-literate users. in Information and Communication Technologies and Development. 2007; ICTD 2007. International Conference on. 2007: IEEE.
23. Haines A, David S, Uta L, Alexander K, Joy E, Steve J. et al. Achieving child survival goals: potential contribution of community health workers. Lancet. 2007; 369: 2121-31.
24. Wilkins K, Nsubuga P, Mendlein J, Mercer D, Pappaioanou M, The data for decision making project: assessment of surveillance systems in developing countries to improve access to public health information. Public Health. 2008; 122: 914-22.
25. Updated Guidelines for Evaluating Public Health Surveillance Systems 2001. [Online] 2001 [cited 2011 April 12]. Available from: URL: http://www.cdc.gov/mmwr/preview/ mmwrhtml/rr5013a1.htm
26. Gazarian M, Williams E, Elliott K, Chant H, Longbottom H, Mellis C, et al. Evaluation of a national surveillance unit. Arch Dis Child. 1999; 80: 21-7.
27. Basu S, Jason A, Sandeep K, Rajesh P, David S. Comparative performance of private and public healthcare systems in low-and middle-income countries: a systematic review. PLoS Med 2012; 9: e1001244.
28. Khan MI, Sahito S, Khan M, Wassan S, Shaikh A, Maheshwari A, et al. Enhanced disease surveillance through private health care sector cooperation in Karachi, Pakistan: experience from a vaccine trial. Bull World Health Organization. 2006; 84: 72-7.
29. Waters H, L Hatt, D Peters. Working with the private sector for child health. Health Policy Planning. 2003; 18: 127-37.
30. Saeed KM, R Bano, RJ Asghar. Evaluation of the national tuberculosis surveillance system in Afghanistan. East Mediterr Health J. 2013; 19: 200-7.
31. Yu P, De Courten M, Pan E, Galea G, Pryor J. The development and evaluation of a PDA-based method for public health surveillance data collection in developing countries. Int J Med Inform. 2009; 78: 532-42.