荷花池汽车站,英国海外志愿服务社,苏代为燕说齐
摘要
本文旨在讨论数据仓库和数据挖掘的概念、数据挖掘和数据仓库的工具和技术以及组织实践的概念。它还包括在当前商业社区中，数据仓库和数据挖掘中的趋势和应用。 关键字
数据库、数据仓库、数据挖掘、数据库管理。 前言
组织在从事日常事务中，使用信息系统来记录和检索数据。信息系统通过数据库链接到更为重要的战略决策，为公司未来的发展提供了更有价值的数据。一个组织可以通过他们所拥有的数据预测即将到来的可能性事件。也可以通过提供数据探索可能性的解决方案，以克服他们所面临的问题,甚至,他们可以使用数据来获得商业环境中的竞争优势。目前，数据库已经减少了,在某些地方,旧的存储方法和保存信息的方式已经消失了,也就是说,传统的文件系统使用方法已经慢慢地得到了改进和发展。数字化时代数据的变化和数据存储库的建立已经创造了一个信息系统的新领域,产生了一些新的职位，并为人们日常生活和工作提供了一个新的生活方式。 Abstract
This paper aims to discuss about data warehousing and data mining, the tools and techniques of data mining and data warehousing as well as the benefits of practicing the concept to the organisations. It also includes the trends and application in data warehouse and data mining in current business communities. Keywords
Database, data warehouse, data mining, database management. Introduction
Organisation uses information systems to record and retrieve data from daily transactions. The information systems via the database that link to it provides valuable data for making important and strategic decisions in regards to the well-being of a company. An organisation can predict the expectation that is yet to come from the data that they possessed. The data can also be used to provide possible solutions to overcome the problems that they faced, and even, they can use the data to obtain competitive advantage in their business environment. Database has reduces, if not in some place, vanish the old method of storing and keeping the information, that is, through the usage of the traditional filing system. The change towards digitization of data and the establishment of data repository has created a new term in the field of information systems, new position in the organisation, and a new way of doing business and daily transactions in human life.
This paper will discuss further about the two terminologies which is data warehouse and data mining from the perspective of database management in the organisation. At the same time, this paper will also include some cases and issues about data warehouse in the organisation according to real situation based on the literatures.
According to William H. Inmon, data warehouse is a set of integrated, subject oriented databases designed to support Decision Support Systems (DSS) functions, where each series of data is precise to some period of time. It is said that data warehouse contains atomic data and lightly conclude the data.
On the other hand, data mining is the search for valuable information in large volumes of data (Weiss & Indurkhya, 1998). It is the process of nontrivial extraction of implicit, previously unknown and potentially useful information such as knowledge rules, constraints, and regularities from data stored in repositories using pattern recognition technologies as well as statistical and mathematical techniques (Technology Forecast, 1997; Piatetsky-Shapiro and Frawley, 1991). As mentioned earlier, many organisations nowadays use computers especially through the usage of information system to collect particulars of business transactions such as records of banking operations, sales of retails, productions of factory, telecommunications and other transactions. Consequently the data mining tools are used to expose positive potentials and association from the data collected. Background of data warehousing and data mining
The following part point up the historical evolution of the database and directly discuss about data warehouse and data mining. A brief history of data warehousing and data mining are included. Furthermore is the issues faced in the early years of implementing the concept of data warehousing and data mining and where both concepts are useful.
Data warehousing started in the late 1980s from the IBM lab and the responsible researchers are Barry Devlin and Paul Murphy. They started by the development of business data warehouse for decision support surroundings. In the early 1990s, it became a trend for organisations to meet the growing demand for organising information.
However Haisten (1999), a columnist for Information Management Website, mentioned that the concept of data warehouse take shape in early 1970s through a study that started out at MIT with the aim to provide optimal technical architecture.
And now, the next generation of data warehousing called Trend in Data Warehouse (TDWI) is mushrooming and become popular in many organisations that use information as their vital capitals.
The emergence of data mining began in the late of 1980s and it flourished by 1990s. There are three roots that can be traced back along three family lines on the origin of data mining, which are the classical statistics, artificial intelligence, and machine learning. In order to automate the process of extracting the data which are increased every single time, human has increased the power of computer and data storage. For that reason, the amount of data becomes huge and more complex. Primarily, Bayes' theorem (1997) and Regression analysis has identify patterns in data. The data mining is actually the process or method by using greater discovering in computer science engineering such as neural networks, clustering process, genetic algorithm and decision trees. Data mining can be said as a method to help with the collection of observation of behaviour.
Ayre (2006) stated in his paper that today's data mining techniques is due to the work of mathematician, logicians, and computer scientist join together to create Artificial Intelligence (AI) and Machine Learning dated back from the 1950s. That was a very basic spark for data mining ideology. As mention earlier, in the 1960s, AI and statistic practitioners created new algorithm such as regression analysis, maximum likelihood estimates, neural networks, bias reduction, and linear model.
Also in 1960s, the field of information retrieval (IR) made its contribution in the form of clustering techniques and similarity measures. At these time techniques were applied to text document, but they would later be utilized when mining data in databases and other large, distributed data sets (Dunham, 2003).
In 1997, Connecticut-based Gartner Group report has mentioned about data mining and artificial intelligence are at the top five ranking of major technology areas that will clearly have a main crash transversely the whole scope of business unit within the incoming three to five years. Presently, data mining techniques and tools are being prolonged to the variety of areas. For instance, the data mining tools like intelligent text-mining system will extract the text waste pertinent to user queries.
The above is the process of how the data is transport to database and data warehouse and selection process by using data mining techniques and technology. And then it show us how the information form by the translating the data to be deploy in business. Approaches of data warehousing and data mining in various industries
The industry of finance, sales and marketing, administration and others should see information as corporate source but the many local narrow systems that held that information simply did not give way the incorporated commercial viewpoint that was required. (Inmon, 2007)
Even though operational data is a greater asset to the organisation, it seemed data is usually not making use to its full capable. Therefore, data warehouse basically is to enable users' appropriate access to breaking apart and complete view of the organisation, supporting forecasting and decision-making process at the managerial stage. Additionally, data warehouse can achieve information consistency by carry data from dissimilar data foundations into centre of database. Users from different department for instances, can view the data from consistent single one place repository. The layer of data in data warehouse makes the information consistent by enable data around the data warehouse to be describe in business terms as against to using database terminology. The establishment of data that enforce how business terms are declared or calculated are also defined in the metadata layer and then served to the users. Because of the data in the data warehouse is non-volatile but it must be design to adapt the changes periodically. It is because terminologies use in business cannot run from changes. Mannino and Walter (2004) in their study about the refreshment of data warehouse stated that data warehouse refreshment is a complex process comprising many tasks, such as extraction, transformation, integration, cleaning, key management, history management, and loading. This study is base on interviewed of 13 organisations and the author conclude that daily refresh during nonbusiness hours were the most common policy. |
