Limited Domains and Internet Protocols The University of Auckland
Huawei Technologies
Today, the Internet does not have a well-defined concept of limited domains. One result of this is that certain protocols and features fail on certain paths. Earlier analyses of this topic have focused either on the loss of transparency of the Internet , or on the middleboxes responsible for that loss , , . Unfortunately the problems persist, both in application protocols, and even in very fundamental mechanisms. For example, the Internet is not transparent to IPv6 extension headers , and Path MTU Discovery has been unreliable for many years , . IP fragmentation is also unreliable , and problems in TCP MSS negotiation have been reported . On the security side, the widespread insertion of firewalls at domain boundaries that are perceived by humans but unknown to protocols results in arbitrary failure modes as far as the application layer is concerned. There are operational recommendations and practices that effectively guarantee arbitrary failures in realistic scenarios . Investigations of the unreliability of IP fragmentation and the filtering of IPv6 extension headers strongly suggest that at least for some protocol elements, transparency is a lost cause and middleboxes are here to stay. In the following two sections, we show that some application environments require protocol features that cannot, or should not, cross the whole Internet.
This section lists various examples of specific limited domain solutions that have been proposed or defined. It intentionally does not include Layer 2 technology solutions, which by definition apply to limited domains. It is worth noting, however, that with recent developments such as TRILL or Shortest Path Bridging , Layer 2 domains may become very large. Differentiated Services. This mechanism allows a network to assign locally significant values to the 6-bit Differentiated Services Code Point field in any IP packet. Although there are some recommended codepoint values for specific per-hop queue management behaviours, these are specifically intended to be domain-specific codepoints with traffic being classified, conditioned and mapped or re-marked at domain boundaries (unless there is an inter-domain agreement that makes mapping or re-marking unnecessary). Integrated Services. Although it is not intrinsic in the design of RSVP , it is clear from many years' experience that Integrated Services can only be deployed successfully within a limited domain that is configured with adequate equipment and resources. Network function virtualisation. As described in , this general concept is an open research topic, in which virtual network functions are orchestrated as part of a distributed system. Inevitably such orchestration applies to an administrative domain of some kind, even though cross-domain orchestration is also a research area. Service Function Chaining (SFC). This technique assumes that services within a network are constructed as sequences of individual service functions within a specific SFC-enabled domain such as a 5G domain. As that RFC states: "Specific features may need to be enforced at the boundaries of an SFC-enabled domain, for example to avoid leaking SFC information". A Network Service Header (NSH) is used to encapsulate packets flowing through the service function chain: "The intended scope of the NSH is for use within a single provider's operational domain." Firewall and Service Tickets (FAST). Such tickets would accompany a packet to claim the right to traverse a network or request a specific network service . They would only be meaningful within a particular domain. Data Centre Network Virtualization Overlays. A common requirement in data centres that host many tenants (clients) is to provide each one with a secure private network, all running over the same physical infrastructure. describes various use cases for this, and specifications are under development. These include use cases in which the tenant network is physically split over several data centres, but which must appear to the user as a single secure domain. Segment Routing. This is a technique which "steers a packet through an ordered list of instructions, called segments" . The semantics of these instructions are explicitly local to a segment routing domain or even to a single node. Technically, these segments or instructions are represented as an MPLS label or an IPv6 address, which clearly adds a semantic interpretation to them within the domain. Autonomic Networking. As explained in , an autonomic network is also a security domain within which an autonomic control plane is used by autonomic service agents. These agents manage technical objectives, which may be locally defined, subject to domain-wide policy. Thus the domain boundary is important for both security and protocol purposes. Homenet. As shown in , a home networking domain has specific protocol needs that differ from those in an enterprise network or the Internet as a whole. These include the Home Network Control Protocol (HNCP) and a naming and discovery solution . Creative uses of IPv6 features. As IPv6 enters more general use, engineers notice that it has much more flexibility than IPv4. Innovative suggestions have been made for: The flow label, e.g. . Extension headers, e.g. for segment routing or OAM marking . Meaningful address bits, e.g. . Also, segment routing uses IPv6 addresses as segment identifiers with specific local meanings . If segment routing is used for network programming , IPv6 extension headers can support rather complex local functionality. The case of the extension header is particularly interesting, since its existence has been a major "selling point" for IPv6, but it is notorious that new extension headers are virtually impossible to deploy across the whole Internet , . It is worth noting that extension header filtering is considered as an important security issue . There is considerable appetite among vendors or operators to have flexibility in defining extension headers for use in limited or specialised domains, e.g. , , and . Locally significant hop-by-hop options are also envisaged, that would be understood by routers inside a domain but not elsewhere, e.g., . Deterministic Networking (DetNet). The Deterministic Networking Architecture and encapsulation aim to support flows with extremely low data loss rates and bounded latency, but only within a part of the network that is "DetNet aware". Thus, as for differentiated services above, the concept of a domain is fundamental. Provisioning Domains (PvDs). An architecture for Multiple Provisioning Domains has been defined to allow hosts attached to multiple networks to learn explicit details about the services provided by each of those networks. Address Scopes. For completeness, we mention that, particularly in IPv6, some addresses have explicitly limited scope. In particular, link-local addresses are limited to a single physical link , and Unique Local Addresses are limited to a somewhat loosely defined local site scope. Previously, site-local addresses were defined, but they were obsoleted precisely because of "the fuzzy nature of the site concept" . Multicast addresses also have explicit scoping . As an application layer example, consider streaming services such as IPTV infrastructures that rely on standard protocols, but for which access is not globally available. All of these suggestions are only viable within a specified domain. Neverthless, all of them are clearly intended for multivendor implementation on thousands or millions of network domains, so interoperable standardization would be beneficial. This argument might seem irrelevant to private or proprietary implementations, but these have a strong tendency to become de facto standards if they succeed, so the arguments of this document still apply.
Noting that limited-domain protocols have been defined in the past, and that others will undoubtedly be defined in the future, it is useful to consider how a protocol can be made aware of the domain within which it operates, and how the domain boundary nodes can be identified. As the taxonomy in shows, there are numerous aspects to a domain. However, we can identify some generally required features and functions that would apply partially or completely to many cases. Today, where limited domains exist, they are essentially created by careful configuration of boundary routers and firewalls. If a domain is characterized by one or more address prefixes, address assignment to hosts must also be carefully managed. This is an error-prone method and a combination of configuration errors and default routing can lead to unwanted traffic escaping the domain. Our basic assumption is therefore that it should be possible for domains to be created and managed automatically, with minimal human configuration. We now discuss requirements for automating domain creation and management. Firstly, if we drew a topology map, any domain -- virtual or physical -- will have a well defined boundary between "inside" and "outside". However, that boundary in itself has no technical meaning. What matters in reality is whether a node is a member of the domain, and whether it is at the boundary between the domain and the rest of the Internet. Thus the boundary in itself does not need to be identified, but boundary nodes face both inwards and outwards. Inside the domain, a sending node needs to know whether it is sending to an inside or outside destination; and a receiving node needs to know whether a packet originated inside or outside. Also, a boundary node needs to know which of its interfaces are inward-facing or outward-facing. It is irrelevant whether the interfaces involved are physical or virtual. To underline that domain boundaries need to be identifiable, consider the statement from the Deterministic Networking Problem Statement that "there is still a lack of clarity regarding the limits of a domain where a deterministic path can be set up". This remark can certainly be generalised. With this perspective, we can list some general functional requirements. An underlying assumption here is that domain membership operations should be cryptographically secured; a domain without such security cannot be reliably protected from attack. Domain Identity. A domain must have a unique and verifiable identifier; effectively this should be a public key for the domain. Without this, there is no way to secure domain operations and domain membership. The holder of the corresponding private key becomes the trust anchor for the domain. Nesting. It must be possible for domains to be nested (see, for example, the network slicing example mentioned above). Overlapping. It must be possible for nodes and links to be in more than one domain (see, for example, the case of PVDs mentioned above). Node Eligibility. It must be possible for a node to determine which domain(s) it can potentially join, and on which interface(s). Secure Enrolment. A node must be able to enrol in a given domain via secure node identfication and to acquire relevant security credentials (authorization) for operations within the domain. If a node has multiple physical or virtual interfaces, they may require to be individually enrolled. Withdrawal. A node must be able to cancel enrolment in a given domain. Dynamic Membership. Optionally, a node should be able temporarily leave or rejoin a domain (i.e. enrolment is persistent but membership is intermittent). Role, implying authorization to perform a certain set of actions. A node must have a verifiable role. In the simplest case, the choices of role are "interior node" and "boundary node". In a boundary node, individual interfaces may have different roles, e.g. "inward facing" and "outward facing". Verify Peer. A node must be able to verify whether another node is a member of the domain. Verify Role. A node must be able to learn the verified role of another node. In particular, it must be possible for a node to find boundary nodes (interfacing to the Internet). Domain Data. In a domain with management requirements, it must be possible for a node to acquire domain policy and/or domain configuration data. This would include, for example, filtering policy to ensure that inappropriate packets do not leave the domain. These requirements could form the basis for further analysis and solution design. Another aspect is whether individual packets within a limited domain need to carry any sort of indicator that they belong to that domain, or whether this information will be implicit in the IP addresses of the packet. A related question is whether individual packets need cryptographic authentication. This topic is for further study.
This document makes no request of the IANA.
. IEEE Standard for Local and metropolitan area networks - Bridges and Bridged Networks HUAWEI - Big IP Initiative.
draft-carpenter-limited-domains-00, 2018-06-11: Initial version draft-carpenter-limited-domains-01, 2018-07-01: Minor terminology clarifications draft-carpenter-limited-domains-02, 2018-08-03: Additions following IETF102 discussions Updated authorship/contributors draft-carpenter-limited-domains-03, 2018-09-12: First draft of taxonomy Editorial improvements draft-carpenter-limited-domains-04, 2018-10-14: Reorganized section 3 Newly written sections 6 and 7 Editorial improvements draft-carpenter-limited-domains-05, 2018-12-12: Added discussion of transparency/filtering debates Added discussion of "controlled environment" Modified assertion about localized standards Editorial improvements draft-carpenter-limited-domains-06, 2019-03-02: Minor updates, fixed reference nits draft-carpenter-limited-domains-07, 2019-04-15: Moved taxonomy to an appendix. Added examples and references. Editorial improvements draft-carpenter-limited-domains-08, 2019-06-12: Added short discussion of address scopes. Added possibility of nested or overlapped domains. Integrated other comments received. Editorial improvements draft-carpenter-limited-domains-09, 2019-06-21: Additional 5G citations. draft-carpenter-limited-domains-10, 2019-08-02: ISE comments. draft-carpenter-limited-domains-11, 2019-10-31: Incorporate review comments. Editorial improvements. draft-carpenter-limited-domains-12, 2019-11-30: Incorporate ISE comments.
This appendix develops a taxonomy for describing limited domains. Several major aspects are considered in this taxonomy: The domain as a whole. The individual nodes. The domain boundary. The domain's topology. The domain's technology. How the domain connects to the Internet. The security, trust, and privacy model. Operations. The following sub-sections analyse each of these aspects.
Why does the domain exist? (e.g., human choice, administrative policy, orchestration requirements, technical requirements such as operational partitioning for scaling reasons) If there are special requirements, are they at Layer 2, Layer 3 or an upper layer? Where does the domain lie on the spectrum between completely managed by humans and completely autonomic? If managed, what style of management applies? (Manual configuration, automated configuration, orchestration?) Is there a policy model? (Intent, configuration policies?) Does the domain provide controlled or paid service or open access?
Is a domain member a complete node, or only one interface of a node? Are nodes permanent members of a given domain, or are join and leave operations possible? Are nodes physical or virtual devices? Are virtual nodes general-purpose, or limited to specific functions, applications or users? Are nodes constrained (by battery etc)? Are devices installed "out of the box" or pre-configured?
How is the domain boundary identified or defined? Is the domain boundary fixed or dynamic? Are boundary nodes special? Or can any node be at the boundary?
Is the domain a subset of a layer 2 or 3 connectivity domain? Does the domain overlap other domains? (In other words, a node may or may not be allowed to be a member of multiple domains.) Does the domain match physical topology, or does it have a virtual (overlay) topology? Is the domain in a single building, vehicle or campus? Or is it distributed? If distributed, are the interconnections private or over the Internet? In IP addressing terms, is the domain Link-local, Site-local, or Global? Does the scope of IP unicast or multicast addresses map to the domain boundary?
What routing protocol(s) are used, or even different forwarding mechanisms (MPLS or other non-IP mechanism)? In an overlay domain, what overlay technique is used (L2VPN, L3VPN,...)? Are there specific QoS requirements? Link latency - normal or long latency links? Mobility - are nodes mobile? Is the whole network mobile? Which specific technologies, such as those in , are applicable?
Is the Internet connection permanent or intermittent? (Never connected is out of scope.) What traffic is blocked, in and out? What traffic is allowed, in and out? What traffic is transformed, in and out? Is secure and privileged remote access needed? Does the domain allow unprivileged remote sessions?
Must domain members be authorized? Are all nodes in the domain at the same trust level? Is traffic authenticated? Is traffic encrypted? What is hidden from the outside?
Safety level - does the domain have a critical (human) safety role? Reliability requirement - normal or 99.999% ? Environment - hazardous conditions? Installation - are specialists needed? Service visits - easy, difficult, impossible? Software/firmware updates - possible or impossible?