close
This essay has been submitted by a student. This is not an example of the work written by professional essay writers.

Issues to Data Integration

downloadDownload printPrint

Pssst… we can write an original essay just for you.

Any subject. Any type of essay.

We’ll even meet a 3-hour deadline.

Get your price

121 writers online

blank-ico
Download PDF

One of the most important tasks in the process of data integration is setting realistic expectations. The term data integration invokes a perfect coordination of diversified databases, software, equipment, and personnel into a smoothly functioning alliance, free of the persistent headaches that mark less comprehensive systems of information management. Think again. The requirements analysis stage offers one of the best opportunities in the process to recognize and digest the full scope of complexity of the data integration task. Thorough attention to this analysis is possibly the most important ingredient in creating a system that will live to see adoption and maximum use.

As the field of data integration progresses, however, other common obstacles and compensatory solutions will be easily identified. Current integration practices have already highlighted a few familiar challenges as well as strategies to address them, as outlined below.

  1. Unrelated Data
  2. For most transportation agencies, data integration involves synchronizing huge quantities of variable, heterogeneous data resulting from internal legacy systems that vary in data format. Legacy systems may have been created around flat file, network, or hierarchical databases, unlike newer generations of databases which use relational data. Data in different formats from external sources continue to be added to the legacy databases to improve the value of the information. Each generation, product, and home-grown system has unique demands to fulfill in order to store or extract data. So data integration can involve various strategies for coping with heterogeneity. In some cases, the effort becomes a major exercise in data homogenization, which may not enhance the quality of the data offered.

  3. Corrupt Data
  4. Data quality is a top concern in any data integration strategy. Given data must be cleaned up prior to conversion and integration, or an agency will almost certainly face serious data problems later. Legacy data impurities have a compounding effect; by nature, they tend to concentrate around high volume data users. If this information is corrupt, so, too, will be the decisions made from it. It is not unusual for undiscovered data quality problems to emerge in the process of cleaning information for use by the integrated system. The issue of bad data leads to procedures for regularly auditing the quality of information used. But who holds the ultimate responsibility for this job is not always clear.

  5. Lack of Storage Capacity Challenges
  6. The unanticipated need for additional performance and capacity is one of the most common challenges to data integration, particularly in data warehousing. Two storage-related requirements generally come into play: extensibility and scalability. Anticipating the extent of growth in an environment in which the need for storage can increase exponentially once a system is initiated drives fears that the storage cost will exceed the benefit of data integration. Introducing such massive quantities of data can push the limits of hardware and software. This may force developers to instigate costly fixes if an architecture for processing much larger amounts of data must be retrofitted into the planned system.

  7. Unexpected Rates
  8. Data integration costs are fueled largely by items that are difficult for the uninitiated to quantify, and thus predict. These might include: Labor costs for initial planning, evaluation, programming and additional data acquisition Software and hardware purchases Unanticipated technology changes/advances Both labor and the direct costs of data storage and maintenance It is important to note that, regardless of efforts to streamline maintenance, the realities of a fully functioning data integration system may demand a great deal more maintenance than could be anticipated.

    Unrealistic estimating can be driven by an overly optimistic budget, particularly in these times of budget shortfall and doing more with less. More users, more analysis needs and more complex requirements may drive performance and capacity problems. Limited resources may cause project timelines to be extended, without commensurate funding. Unanticipated issues, or new issues, may call for expensive consulting help. And the dynamic atmosphere of today’s transportation agency must be taken into account, in which lack of staff, changes in business processes, problems with hardware and software, and shifting leadership can drive additional expense. The investment in time and labor required to extract, clean, load, and maintain data can creep if the quality of the data presented is weak. It is not unusual for this to produce unanticipated labor costs that are rather alarmingly out of proportion to the total project budget.

  9. Lack of Cooperation from Staff
  10. User groups within an agency may have developed databases on their own, sometimes independently from information systems staff, that are highly responsive to the users’ particular needs. It is natural that owners of these functioning standalone units might be skeptical that the new system would support their needs as effectively. Other proprietary interests may come into play. For example, division staff may not want the data they collect and track to be at all times transparently visible to headquarters staff without the opportunity to address the nuances of what the data appear to show. Owners or users may fear that higher ups without appreciation of the peculiarities of a given method of operation will gain more control over how data is collected and accessed organization-wide. In some agencies, the level of personnel, consultants, and financial support emanating from the highest echelons of management may be insufficient to dispel these fears and gain cooperation. Top management must be fully invested in the project. Otherwise, the likelihood is smaller that the strategic data integration plan and the resources associated with it will be approved. The additional support required to engage and convey to everyone in the agency the need for and benefits of data integration is unlikely to flow from leaders who lack awareness of or commitment to the benefits of data integration.

  11. Lack of Data Management Expertise
  12. As more transportation agencies nationwide undertake the integration of data, the availability of experienced personnel increases. However, since data integration is a multi-year, highly complex proposition, even these leaders may not have the kind of expertise that evolves over a full project life-cycle. Common problems develop at different stages of the process and these can better be anticipated and addressed when key personnel have managed the typical variables of each project phase. Also, the process of transferring historical data from its independent source to the integrated system may benefit from the knowledge of the manager who originally captured and stored the information. High turnover in such positions, along with early retirements and other personnel shifts driven by an historically tight budget environment, may complicate the mining and preparation of this data for convergence with the new system.

  13. Perception of Data Integration as an Overwhelming Effort
  14. When transportation agencies consider data integration, one pervasive notion is that the analysis of existing information needs and infrastructure, much less the organization of data into viable channels for integration, requires a monumental initial commitment of resources and staff. Resource-scarce agencies identify this perceived major upfront overhaul as “unachievable” and “disruptive.” In addition, uncertainties about funding priorities and potential shortfalls can exacerbate efforts to move forward.

infoRemember: This is just a sample from a fellow student.

Your time is important. Let us write you an essay from scratch

100% plagiarism-free

Sources and citations are provided

Find Free Essays

We provide you with original essay samples, perfect formatting and styling

Cite this Essay

To export a reference to this article please select a referencing style below:

Issues to Data integration. (2019, February 27). GradesFixer. Retrieved July 29, 2021, from https://gradesfixer.com/free-essay-examples/issues-to-data-integration/
“Issues to Data integration.” GradesFixer, 27 Feb. 2019, gradesfixer.com/free-essay-examples/issues-to-data-integration/
Issues to Data integration. [online]. Available at: <https://gradesfixer.com/free-essay-examples/issues-to-data-integration/> [Accessed 29 Jul. 2021].
Issues to Data integration [Internet]. GradesFixer. 2019 Feb 27 [cited 2021 Jul 29]. Available from: https://gradesfixer.com/free-essay-examples/issues-to-data-integration/
copy to clipboard
close

Sorry, copying is not allowed on our website. If you’d like this or any other sample, we’ll happily email it to you.

    By clicking “Send”, you agree to our Terms of service and Privacy statement. We will occasionally send you account related emails.

    close

    Attention! This essay is not unique. You can get a 100% Plagiarism-FREE one in 30 sec

    Receive a 100% plagiarism-free essay on your email just for $4.99
    get unique paper
    *Public papers are open and may contain not unique content
    download public sample
    close

    Sorry, we could not paraphrase this essay. Our professional writers can rewrite it and get you a unique paper.

    close

    Thanks!

    Your essay sample has been sent.

    Want us to write one just for you? We can custom edit this essay into an original, 100% plagiarism free essay.

    thanks-icon Order now
    boy

    Hi there!

    Are you interested in getting a customized paper?

    Check it out!
    Having trouble finding the perfect essay? We’ve got you covered. Hire a writer
    exit-popup-close

    Haven't found the right essay?

    Get an expert to write you the one you need!

    exit-popup-print

    Professional writers and researchers

    exit-popup-quotes

    Sources and citation are provided

    exit-popup-clock

    3 hour delivery

    exit-popup-persone