An Introduction to Ontology Integrated EA 2008 Chris Partridge Chief Ontologist Tutorial - purpose Hopefully, the tutorial, should help you to explain what an ontology is and how it can be useful identify several common misunderstandings when using attempting to ‘do’ ontology see why a top ontology is useful, and appreciate its technical nature become acquainted with one example top ontology – BORO It is intended to help you understand ontology, it is not intended to turn you into an ontologist 2 Caveat – area of interest Large Operational Enterprise Systems ERP Systems CRM Systems FX Settlement Systems Trading Accounting Systems Retail POS Systems Logistics Systems Air Defence Systems Semantic Web Applications “Collective Knowledge” Systems Social networking – Facebook Wikis Bio-medical dictionaries Inference Logics – first order, description 3 Structure of the tutorial A brief history of ontology Example use - semantic interoperability Common misunderstandings Why you need a Top Ontology Choosing the right top ontology Using the ontology for semantic alignment 4 A Brief History Of Ontology A brief history of ontology History of the word the word ontology is from the Greek ὄν , genitive ὄντος : of being (part. of εἶναι : to be) and - λογία : science, study, theory While the etymology is Greek, the oldest extant record of the word itself is the Latin form ontologia , which appeared in 1661, in the work Ogdoas Scholastica by Jacob Lorhard ( Lorhardus ) and in 1631 in the Lexicon Philosophicum by Rudolph Göckel ( Goclenius ). By this stage, it is regarded as forming the basic subject matter of metaphysics 6 Ontology – as philosophy Origins ontology as a mode of analysis is generally thought to have originated in early Greece and occupied most famously Aristotle, who created the first system of ontology in the form of an ontology of substances – often represented pictorially in the tree of Porphyry 7 Ontology in the 1960s and 1970s Connection with Ontology and ‘Reality’ recognised from the start the issue is ontology, or the question of what exists. (Mealy 1967. p. 525) for some time now my work has concerned the representation of information in computers. The work has involved such things as file organizations, indexes, hierarchical structures, network structures, relational models, and so on. After a while it dawned on me that these are all just maps, being poor artificial approximations of some real underlying terrain. (William Kent 1978, “Data And Reality: Basic Assumptions in Data Processing Reconsidered”) Resulting view codified / standardised in the early 80s ANSI-SPARC - Griethuysen , J.v . ISO/TC97/SC5/WG3-N695 - Concepts and Terminology for the Conceptual Schema and the Information Base., ANSI, New York, NY, 1982 8 View in the early 1980s UoDD (data) is a description of the UoD (ontology) Reference: Griethuysen , J van, "ISO/TC97/SC5/WG3-N695 - Concepts and Terminology for the Conceptual Schema and the Information Base.," ISO/TC97/SC5/WG3-N695, ANSI, 1982 9 George Mealy: in more detail Mealy* distinguishes three distinct realms of interest the real world itself ideas about it existing in the minds of men symbols on paper or some other storage medium the latter realms are, in some sense, held to be models of the former. Thus, we might say that data are fragments of a theory of the real world, and data processing juggles representations of these fragments of theory. No one ever saw or pointed at the integer we call “five” – it is theoretical – but we have all seen various representations of it, such as V 101 2 5 8 5 5 and we recognize them all as denoting the same thing, with perhaps different flavours. …The issue is ontology, or the question of what exists. (Mealy 1967. p. 525) *Mealy, GH 1967 “Another Look at Data,” Proceedings of the Fall Joint Computer Conference, November 14–16, Anaheim, California 10 Bill Kent: in more detail A message to mapmakers: highways are not painted red, rivers don't have county lines running down the middle, and you can't see contour lines on a mountain For some time now my work has concerned the representation of information in computers. The work has involved such things as file organizations, indexes, hierarchical structures, network structures, relational models, and so on. After a while it dawned on me that these are all just maps, being poor artificial approximations of some real underlying terrain These structures give us useful ways to deal with information, but they don't always fit naturally, and sometimes not at all. Like different kinds of maps, each kind of structure has its strengths and weaknesses, serving different purposes, and appealing to different people in different situations. Data structures are artificial formalisms. They differ from information in the same sense that grammars don't describe the language we really use, and formal logical systems don't describe the way we think. "The map is not the territory" [Hayakawa] 11 Defining ontology – in philosophy 20th Century views Quine claimed that the question ontology asks can be stated in three words ‘What is there?’ and the answer in one ‘everything’ not only that, “everyone will accept this answer as true” though “there remains room for disagreement over cases.” (“On What There Is”) Mealy refers to this essay of Quine’s Jonathon Lowe has a more technical definition an ontology is “the set of things whose existence is acknowledged by a particular theory or system of thought.” (E. J. Lowe, The Oxford Companion to Philosophy) 12 Defining ontology - in enterprise systems Enterprise systems can be seen as ‘theories’ of their domains ( Naur 1985) Recasting the philosophical description in these terms Enterprise System ontology: The set of things whose existence is acknowledged by a particular enterprise system A common way of characterizing this ‘acknowledgment’ relationship is as one of ‘ ontic commitment’. ( Quine 1969) Naur , P, “Programming as Theory Building”, Microprocessing and Microprogramming, 15, (1985), 254-261 Quine , WV, Ontological relativity, and other essays, Columbia University Press, New York, 1969 13 Ontology – the one ‘Real World’ Branch System Head Office System Datawarehouse Application Views Domain 14 Real World Objects 15 style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h Ontology as an engineering artefact For the ontology to be useful, to be an engineering artefact, we need to describe it In AI, the term ‘ontology’ tends to be used to refer to the description (the engineering artefact), rather than what the engineering artefact describes One can then characterise the way it is described (independently of what or how it describes) “An ontology consists of a specific vocabulary, plus a set of explicit assumptions in the form of a first-order logical theory, where vocabulary words appear as unary or binary predicate names” One can then develop ontology languages that enforce these ways of describing if one wishes to do reasoning, then it makes sense to take this route 16 Two aspects Ontology and the real world what things exists? Ontology as an engineering artefact how is the description artefact structured? What language is used? Both aspects need to be formalised For enterprise systems, formalising the first aspect is important for semantic interoperability, it is essential no amount of work on the second aspect, by itself, can get to the heart of semantic interoperability 17 Example Use - Semantic Interoperability An area where ontology is useful Semantic interoperability Ontologies have a number of uses A good example is semantic interoperability 19 style.visibility ppt_w ppt_h Interoperability is a current issue Emerging automation: occupying the automatable space 20 Common Misunderstandings When trying to ‘do’ ontology Ontology anti-patterns 22 style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h Ontology anti-patterns Look at two anti-patterns anti-real- worldism various strategies for ignoring the real world real-real- worldism superficial-real- worldism assuming that everyone knows intuitively what exists – hence there is no need to work at, or plan for, identifying this deep-real- worldism 23 Anti-Real- Worldism Just look at the data, do not worry about what it represents Anti-real-worldism One aspect of ontology focuses on what exists, what the data in a system represents Anti-real- worldism a variety of positions that result in a focus on the data rather than the ‘real world’ for some time now my work has concerned the representation of information in computers. The work has involved such things as file organizations, indexes, hierarchical structures, network structures, relational models, and so on. After a while it dawned on me that these are all just maps, being poor artificial approximations of some real underlying terrain. (William Kent 1978, “Data And Reality: Basic Assumptions in Data Processing Reconsidered”) Similar issue in philosophy semantic ascent versus semantic descent it [semantic ascent] is the shift from talk of miles to talk of 'mile‘: it is the shift from talking in certain terms to talking about them by analogy, semantic descent is the shift from talk of ‘mile’ to talk of miles: it is the shift from talking about certain terms to talking what they refer to 25 Assume data reflects the real world (exactly) System 1 Anti Pattern - Data reflects the real world exactly. Benefit - If true then inter-operability is easy since there would be a 1:1 relationship between data and the real world. Problem - We know this is not true. a x a x System 1 x System 2 Data to Real World Relationship Data to Data Path Syntax Translator 26 Assume data reflects the real world (exactly) In France “Unknown Aircraft” means no identification attempted. In US “Unknown Aircraft” means all attempts to identify failed and therefore if it approaches it can be treated as hostile. Exactly the same data, completely different meaning, with disastrous consequences. a French System US System Data to Real World Relationship Data to Data Path Syntax Translator “Unknown Aircraft “Unknown Aircraft” b 27 Assume data does NOT reflect the real world (exactly) In the real world (and its representation, the ontology) the French data “Unknown Aircraft” maps onto a different object to the US data “Unknown Aircraft” Pro Pattern 1 – Real-Real- Worldism Benefits – Can deal with situations where the mapping between systems in not 1:1 a French System US System Semantic Information Path Real World to Model Relationship “Unknown Aircraft “Unknown Aircraft” b Semantic Interoperability Engine a r b r t r 28 Anti-real-worldism Other varieties all we need to do is formalise our data – for example: translate it into OWL this does not tell us what the data represents we do need to change the data – it is sufficiently rich but is it clear what the data represents? 29 ADatP3 – ACO – A real example There can be quite an intricate implicit real world structure. 30 BORO Analysis Date Time Transaction Type Transaction Number Currency _ 1 Amount _ 1 Counterparty Currency _ 2 Amount _ 2 1 / 1 / 2007 13 : 55 : 0 SpotFX 20070101 - 1 GBP 1 , 0 , 0 CitiBank USD 2 , 0 , 0 Legacy System Table Real-real-worldism – an example 31 style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h Located At Pattern Instance Name Pattern Happens In Pattern Asset Exchange Pattern Located At Pattern Located At Pattern Instance Happens In Pattern Name Pattern Instance Asset Exchange Pattern Instances BORO Active Universal Business Patterns Client Active Business Patterns Client Instance Information Real-real-worldism – an example 32 style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h Superficial-Real- Worldism Superficial-real-worldism Assumes that what objects exist is clearly and unequivocally understood if there are disagreements, these are down to lack of knowledge of the subject matter so no need to formalise or clarify what we mean when we say an object exists There are well-known cases, that undermine this position 34 Superficial-real-worldism Argue that mostly subject matter experts do not have the technical ontological knowledge to draw up a list of what exists that drawing up this list, in a sufficiently precise and formal way, involves some technical ontological work This implies that, in some sense, the list is not in the data. That we use this data to build the list/ontology For those who like technical terms, this is often called a ‘rational reconstruction’. If the people who built the original system we trained in ontology and tried to build one, this is, most likely, what they would end up building Illustrate this is a well-known case 35 Antique example - Ship of Theseus 36 style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h Criterion of identity Issue criterion of identity what basis do I have for saying one ship, sock, axe or broom is identical to another? is there a different basis for different types of things? Solution an extensional criterion of identity for physical objects only one object can occupy the same 4-dimensional space if two objects are different, then there must be some point in space and time which is occupied by one of the objects and not the other 37 Situation - changing chairman 38 Solution – a 4D chairman Apply the criterion of identity 39 Solution – a 4D chairman 40 Why You Need a Top Ontology Why you need to make metaphysical choices Example: top ontology It is helpful if the top ontology is comprehensive if it lists all the kinds of things that exist 42 Criteria of identity – top ontology If you have a list of all the kinds of things that exist, then you can ask whether they have a common criterion of identity, and what it is In the case of BORO each major ontological category has its criteria of identity Things = 4D extension Classes = instances tuples = objects in places 43 Simplified BORO method Start with a term Consider what object the term refers to Ask what the object’s ontological category is Element, Type or tuple If the object is NOT an Element, and we do not know its composition, then identify its composition If it is a type, identify a couple of its instances if these are tuples , identify its places and go to 3 if these are not tuples go back to 3 If the object is an Element or we know its composition, check it conforms to its identity criterion Elements = 4D extension Types = instances tuples = places Stitch the object into the BORO Model Check that the object inherits all characteristics of its Type in the model Return to 1 44 Ontology as a technical discipline Criteria of Identity are clearly technical. 4D objects are also technical hopefully, this illustrates that this kind of work involves what we earlier called ‘rational reconstruction’ Criteria of identity are part of a bigger picture – technically called metaphysical choices the reason for the name is that the choice seems to be independent from any empirical checks on the ‘real world’ however, it looks as if the choice you make can affect how easy it is to build your ontology – so it is worth getting right Brief overview of the metaphysical choices 45 Metaphysical choices A list of some choices extensionalism versus non- extensionalism – I – Universals extensionalism versus non- extensionalism – II – Particulars Second Order Universals perdurantism versus endurantism presentism versus eternalism absolute versus relative space, time and space-time modally extended versus unextended individuals materialism and non-materialism topology of time – branching or linear. See Partridge, C. (2002). LADSEB-CNR - Technical report 06/02 - Note: A Couple of Meta-Ontological Choices for Ontological Architectures 46 BORO’s choices Extensionalism – I – Universals Extensionalism – II – Individuals Higher order universals Perdurantism Eternalism Relative space-time Modally unextended individuals Materialism Topology of time –linear 47 Making metaphysical choices They are not independent, so need to be made consistently Choices need to reflect the enterprise system ontology’s (engineering) goals 48 Choosing The Right Top Ontology Basis for making the choices We call these ‘types of sophistication’ Based upon the notion of what makes a good scientific theory – so have a pedigree Normally list six characteristics generality . The degree by which the scope of the types in the improved model can be increased without the loss of information simplicity . The degree by which the model can be made less complex explanatory power . The ability of the improved model to give increased meaning fruitfulness . The degree to which the improved model can meet currently unspecified requirements or is easily extendable to do so objectivity . The ability of the model to provide a more objective (shared) understanding of the world : in particular, to index a thing to its mode of existence as opposed to its mode of representation and/or application precision . The ability of the improved model to give a more precise picture of the business object These types are closely inter-related In our experience, the result is the identification of very general (highly re-usable) business patterns 50 Generalisation Example and Benefits The stages of generalisation Original Classes Construct Super-Classes Eliminate Original Classes 52 Example : classes of animals (1) – starting point We start with the Animals class having 6 sub-classes Animals Boars Sows Ewes Mares Stallions Rams 53 Example : classes of animals (2) – generalised classes After the re-engineering the original classes have been eliminated overall number of classes reduced new classes fit into two new patterns; species and gender addition of new elements to one pattern, automatically inherit the other pattern (consider – chicken (hen and cockerel)) 54 Pigs Male animals Female animals Animals Horses Sheep SPECIES GENDER Multiple classification’s potential for generalisation T he potential increases substantially as the number of general classes gets higher. This is because the potential number of lower level classes that can be superseded grows exponentially T he figures give an indication of how multiple classification’s potential for generalisation affects increases in scope and functionality. Assume that we double the number of objects in a totally generalised system. If we take the potential number of lower level classes as an indication of the scope and functionality of the system classes, then this doubling of size much more than doubles the scope and functionality. For instance, when we double a system with 10 objects, we should have a theoretical 1000 fold (1048550/1013) increase in scope and functionality. For a larger system, the increase would be even higher 55 Number of general classes 1 2 3 4 5 10 20 100 200 N Potential number of lower level classes 0 1 4 11 26 1013 1,048,555 1.27*10 30 1.61*10 60 2 N -N-1 Twelve high level business patterns A selection of high level patterns from the BDM 56 style.visibility ppt_w ppt_h Drill down into the twelve patterns Concepts balance concept US$ position Partial Identity Whole, part, connections Counterparts (relations between possible and actual) Partial delivery Event Participation Participation relationship Party to contract Temporal Stages Boundaries, stages, Splits, mergers $100 Balance Successions/successors FX Settlement leg Typed Types Object-classification and classification-classification FX Deal Types Tuple types Fixed, variable, couples, triples, quadruples Party to contract Intentionality Intentionally constructed objects FX Contract Universes Possible and actual Failed delivery 57 Scope of functionality to complexity Typically we find that many of the business objects we construct when re-engineering one part of the system are sufficiently general to be re-used many times in other parts. This means that each business object is re-used to do jobs originally done by a number of objects. This has a remarkable effect; as the scope of the re-engineering grows, the model becomes much simpler without losing any power. We call this process “compacting” Also we find that the business objects are general enough to be re-used, with no extra effort, to do things that the current system could not. They become not only simpler, but functionally richer. Systems developed this way better capture the essence of the business 58 Narrower Wider Scope/Functionality Complexity/Size Lower Higher TOO COMPLEX Ontology sophistication based modelling Traditional modelling Example of General Patterns General asset exchange pattern Ontological analysis has revealed a general asset exchange pattern that involves the exchange (of ownership) of assets – this covers security purchases security sales dividend entitlements bonus and rights entitlements tax entitlements stock borrowing and lending agreements term deposits placed and accepted foreign exchange deals call/notice deals 60 A (partial) generalised assets hierarchy The asset super–sub-class hierarchy is rich, with a variety of sub-classes that share in the overall asset pattern This includes things such as dividend entitlement coupons and tax credit vouchers 61 Assets Equities Bonds Pounds Sterling US Dollars Gold Oil Real Estate Commodities Securities Currencies Financial Assets Physical Assets Bond Equity Security Currency Property Commodity Asset A generalised deals hierarchy This shows part of the asset exchange hierarchy relating to deals It shows how the kinds of deal we normally list can be seen as combinations of more general deals. For example, how a stockloan can be seen as a call/notice deal where the asset is a security The implication of this for processes – if delivery can be described in a general way it only needs to be done once for all types. Potential to deliver ‘abundant functionality’ – i.e. substantially more than currently required. So as market conditions change – in many case, the new requirements are already met 62 Deals Asset Exchanges Term Deposits Call/Notice Deposits Stockloans Repos Security Deals Currency Deals Term Deals Call/Notice Deals A generalised accounts hierarchy This shows part of the asset agreement hierarchy that relates to accounts It again shows how the traditional types of accounts are combinations of more general types It also begins to show the wide scope of the accounts pattern. This not only covers the more traditional call/notice deposit and stock depot accounts, it also covers investment portfolios and foreign exchange ( fx ) trading books Note – though only hinted at here, generalisation has revealed how both accounts and exchanges fall under the general asset agreement pattern 63 Accounts Asset Agreement Call/Notice Deposits Investment Portfolios Stockloans Depot Accounts Security Accounts Currency Accounts Deposit Accounts Managed Accounts FX Trading Books Safe Custody Accounts BORO process flow Current Systems (inc. People & Processes) Semantic Alignment Rules Client Model of Business Universal Model of Business BORO Foundation Stitches into Stitches into BORO Foundation 42 Universal Model Of Business Client Model Of Business Client Instances GUI Service #1 Service #2 Service # n Stitches into Stitches into Stitches into Model Transformer Information Transformer Model Transformer Model Transformer Legacy Data Client Model of Business 42 Universal Model of Business Configuration Applications 42 S B S Core Current Systems Legacy Data Data into Information BORO Process Legacy Information Semi-Manual Automatic 64 style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h Semantic Alignment Using the ontology for semantic alignment Semantic analysis process Input Output 66 Semantic cleansing Current State - Models, Standards and Legacy Data. Output Input Semantic Information - Models, Standards and Information. Identifies the underlying semantics. Removes the syntactic ‘dirt’ from the data. 67 style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h Semantic stitching Identify common objects in the model. Framework for a semantically integrated model. Implemented in the 42BDM (and 42SBS Core) as additional names . 68 style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h Semantic stitching Stitch in the new location objects (and names) into an integrated model. Foundation for semantically integrated location model. 69 style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h Basic semantic analysis - vertical match/mismatch Systematic process applied to both models and data Matches - confirm information exists. Mismatches - identify gaps and how they can be filled, both: Downwards, in the applications and Upwards in the models 70 Basic semantic analysis - horizontal match/mismatch Horizontal Mismatches Horizontal Match Can use Client BDM – 42 BDM to identify horizontal matches and mismatches across applications. Identifying matching and missing information (requirements) 71 style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h style.visibility ppt_w ppt_h Some simple examples of the issues discovered Identity Cross Table identity E.g. the same organisation can be a customer and a vendor Process Identify where it can happen – across which tables Identify whether this is actually an issue Identify whether (and how) it is managed Temporality Change over time E.g. Montenegro is about to become a country Process Identify where this is not catered for in the system Identify whether this is actually an issue Identify whether (and how) it is managed 72 style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility Semantic Alignment - Examples Client model stitched into BDM Left panel uses the BORO Names Right panel uses the ISO 15926 Names Greyed out names are where a name of the selected type does not exist Can see how the two ontologies overlap and interleave. Enables inspection – a form of quality assurance 74 style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility style.visibility Client application #1 - stitched into BDM Left panel uses the BORO Names Right panel uses the Location DDM Names Greyed out names are where a name of the selected type does not exist Can see how the two ontologies overlap and interleave Enables inspection – a form of quality assurance 75 Cleansing and integrating the data schemas 76 Application #1 data schemas NB: Values translated as entities (see Business Objects) – i.e. different foundations Application #2 data schemas Cleansing and integrating the data Application #1 data Application #2 data 77 Cleansing and integrating the data 78 Application #2 data Country_Region Wholes-Parts Cleansing and integrating the data 79 Core Loaded Data - Naming Questions Discussion