Apidays Helsinki & EC panel on Sep 25th, 2020 - Chat relevant topics
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Apidays Helsinki & EC panel on Sep 25th, 2020 – Chat relevant topics
Lifecycle and management of versioning of APIs
An interesting discussion was taken about the management of APIs.
- Erik Wilde specifies that there’s a lot of work on API guidelines from companies defining
best practices of which some of them are publicly available
(https://dret.github.io/guidelines). Marjukka Niinioja noticed that "API funnel" also in a
public organization for the behaviour design pattern, too (https://www.apiopscycles.com).
- Erik Wilde commented on API versioning by saying that it is most important to teach
people how to *not version* APIs (https://www.youtube.com/watch?v=sbXEiLrbinE) and
specifically point to Semver as the versioning scheme
(https://www.youtube.com/watch?v=MNU1PimVXMM ). He added that it’s useful to
have deprecation and sunsetting models being part of every API from the very start (no API
lives forever). Marjukka Niinioja pointed to a reference to the versioning and other design
points (https://www.apiopscycles.com/rest-api-design-guide ) (which could have Semver
specifically mentioned too).
- Bert Van Nuffelen informed that in Belgium and Flanders the REST API guidelines take only
the major version in the URL of the API. Semver is added as extra header to the HTTP
message. Roberto Polli said that for Italy that same approach is used. They thought to use
media-type versioning but it would have created more confusion.
- Erik Wilde suggested to break the resources when you break the API and that make
changes discoverable across backwards-compatible changes of the API.
Organisational aspects took also a big part of the discussion:
- Erik Wilde observed that it was very interesting to see “non-tech API users” being targeted.
And proposed to try to add info on each of these about which apps/experiences are
powered by this API.
- Lorenzino Vaccari discussed about the link between applications and APIs and that some
cases were investigated by the JRC
(https://joinup.ec.europa.eu/collection/api4dt/document/science-policy-report-apis-
governments-why-what-and-how ). They have analysed the structure of the
ProgrammableWeb directory (that links APIs, developers, apps, among the others) and of
some public sector solutions (e.g. the national French catalogue presented in this event by
Patrick Amarelis and the ones proposed by the Regione Lombardia that connects API and
apps (https://www.e015.regione.lombardia.it/site/api-catalog ).Erik Wilde said that the
difference is that open data is a product, APIs are a service. Some very different design and
delivery mechanics around it. He agrees with Lorenzino about design and users being
central for the success of this.
- Bert Van Nuffelen asked if API data pass also works for B2B connections. In Flanders there
are 2 security systems in parallell: one C2B (person 2 API) and B2B (for e.g. utility sector).
They are integrating them into one approach, but that will take a time. Marjukka Niinioja
highlighted the B2B question is very important and referenced to Mika Honkanen that
might have something to say about the Finnish model. Erik Wilde added that many of this
can be addressed with a very explicit focus on consumers and UX. Moreover, terrible
processes are not much better when they become terrible digital processes. But, on the
other hand, API initiatives should not try to ‘boil the ocean’.
- Regarding architectures based on microservices, Erik Wilde specifies that in many cases,
microservices is used as a proxy to "let's move from monoliths to something more agile". Inmost cases, the specific tech stack probably should be more modest that going straight to
containerization. The real wins are in decoupling capabilities, not in any specific tech stack.
He also specified that as long as these departments have individually deployable solutions,
they would be "microservices", even though they often may be not so micro at all.
Marjukka Niinioja mentioned a research on the topic: “Bosch, J. – Bosch-Sijtsema, P.
(2010). From integration to composition: On the impact of software product lines, global
development and ecosystems. Journal of Systems and Software, 83(1), s. 67–76.”
- Erik Wilde mentioned that it would be good to very strictly separate service reuse (APIs)
and the possible sharing of IT assets. mMixing them quickly creates coupling that will hold
you back, even if you do start having APIs in some places.
- Bert Van Nuffelen asked if all APIs in a marketplace should satisfy the same guidelines and
added that in Flanders the standardization program Open Specififcations for Linked
Organisations (data.vlaanderen.be) implements the EIF starting from the semantical
agreements. In this context, the markplace discussion is now started. What they actually
bring in a 3helics approach, is making people together around the table on one topic
(industry, government and research).
Discoverability of APIs and standards
Protocols for APIs were among of the main discussed standards in the chat:
- (Erik Wilde) The two main optimizations that HTTP/2 introduces were mentioned. They are
multiplexing and header compression. Regarding the former one, HTTP1.0 protocol is
based on simple model where a client made a request and a server answer to this single
request. HTTP1.1 introduced ‘pipeline’ where. Roughly speaking, a client can send multiple
requests in sequence and the server will answer to these requests in the same order. To
improve HTTP1.1 and solve some issues, such as for example the head-of-line blocking, in
2015, with HTTP/2 multiplexing was introduced. With multiplexing each request goes on its
own in a stream and the server send a response in the same stream. Header compression
is the second big improvement introduced by HTTP2. It the wire format is binary, that can
be compressed and is optimized when same multiple messages are sent. To advance with
HTTP and APIs, the IETF will likely have a new WG about HTTP APIs that, hopefully will
speed up development and delivery of API-level specs on top of HTTP.
- (Marjukka Niinioja) IoT devices should interact via dedicated protocols such as MQTT and
gRPC (which is is even faster), but not suitable for all occasions. Generally these are useful
for known server-side implementation. Erik Wilde added that, "event-based" is a *very*
wide category with very different semantics. AsyncAPI creates this illusion that "hey, it's all
covered by one format", but things are not quite as simple.
- (Roberto Polli) Some considerations for government sector on APIs beyond HTTP have also
been added: services provided to a broad audience ensuring a clear semantic and auditing.
The choice of protocols should be done carefully. E.g. with websockets, which are HTTP-
initiated, there’s no clear semantic after the communication is initiated. This means it
requires a very complex work to implement all those features, which comes for free in
HTTP. (https://forum.italia.it/t/graphql-and-the-italian-interoperability-framework/11762).
Also. while MQTT suffers of a similar semantic issues. iirc AMQP provides some
functionalities to implement semantics in queues. This was also agreed by Erik Wilde that
specified how recent approaches sell some "API metalayer" approach, but that he is quite
sceptical that these work very well unless you're willing to constrain yourself in a variety of
ways.
A deep and participated discussion was made about API metadata specifications.- Lorenzino Vaccari introduced the topic about the use of the OpenAPI specifications for
RESTful APIs and AsyncAPI for even-driven architectures. Erik Wilde clarifies that both
OpenAPI and AsyncAPI don’t have very rich metadata models and that they are very
technical and strictly just describe the interface.
- Bert Van Nuffelen mentioned that in Belgium and the region of Flanders semantic data
standards, and technical standards (e.g. REST API guidelines) are being created. They
covers all aspects from policy to technical alignment with
developers(https://www.gcloud.belgium.be/rest/; https://github.com/belgif;
https://data.vlaanderen.be/standaarden/ ). In particular, in the region of Flanders, just
recently a standardization track has been started on metadata for services
(https://data.vlaanderen.be/standaarden/standaarden-in-ontwikkeling/metadata-voor-
services/index.html ) and DCAT 2.0 profile to capture the wide variety of APIs (geo, SOAP,
REST, Linked Data, ...) into one catalog. With this profile the intent is to enable every
organisation to expose metadata of their API in a standardized way so that a single
marketplace for APIs can be created. Erik Wilde noticed that it would be very useful if this
can be embedded in OpenAPI and AsyncAPI so that it simply can be harvested from API
descriptions. Marjukka Niinioja added that this is like the "product definition" that some
API management solutions have had as proprietary format for some time. Definitely
something for us suggest in OpenAPI Initiative. Bert Van Nuffelen also mentioned that part
of the standardization track is indeed to create addition agreements for OpenAPI so that
the expected metadata is extractable and Erik Wilde referenced an online work they were
working on some time ago (http://dret.net/lectures/api-days-paris-2019/) is but we never
created a proper specification. Roberto Polli added that in Italy they have similar goals and
started discussing within OAS community and Bert Van Nuffelen further specified that in all
these alignment initives (e.g. REST API guidelines) developers & architects want also to
make semantical agreements. So it is a wide space and it is not always easy to make an
independent decision at the technical level.
- About extensibility of OAS Erik Wilde specifies that OAS can support it but that it would be
great to not limit these efforts to OAS. Ideally, it should be something that can stand alone
as well. Marjukka Niinioja noticed that OpenAPI 3.0 has in general better support for that
than 2.0, (https://www.youtube.com/watch?v=InoAIgBZIEA ) and gave a reference to make
questions (or add some answers) on OpenAPI or other specs (https://github.com/api-
specification-toolbox/questions ). Roberto Polli offered collaboration to organisations
interested in reusing or contributing to OAS validation rules. He also specifies that, except
for some casing+metadata, validation rules are based on the HTTP RFCs and can be safely
adopted by other agencies. They are even working on a set of optional security rules, and
he is available to work together on that.
- Levente Ban said that they are getting involved in contributing OAS tools. And he was
curious of the tools/languages used. Roberto Polli asked about which tools were
contributed and specifies that they also that, when OAS3 was lagging, they contributed to
some tools to support moving away from Swagger2. Levente specified that in some areas
they are trying to move to openapi-generator with java. Here the oas v3 with oauth2
support for java is a big need. They are trying to contribute the openapi-generator
functionalities related to this for the javaspring lang. They are also trying to get API first
approach)
- Bert Van Nuffelen added that when changing specifications, budget must be taking into
account, too. If clients of governments are the cities, and the API of the regional/federalgovernment are moving too quickly, they cannot follow. They simply do not have the
manpower + budget to follow these changes.
- Mika Honkanen raise the topic about the need of metadata model (like DCAT Application
Profile - DCAT-AP and https://www.programmableweb.com/ ) to describe APIs and SLAs.
Erik Wilde suggested to check out also schema.org, maybe the ontology that is used for
most data models on the planet, and that it has a proper class about “WebAPIs" in their set
of schemas (https://schema.org/WebAPI). Bert Van Nuffelen informed that more related
to this discussions is BREG DCAT-AP. That specification is intended to describe in full base
registries (thus including relationship with APIs for a base dataset). Bert Van Nuffelen
added that DCAT-AP takes a more wider aim: the standard wants to give an overview of all
(public government) data and its access to it in one catalogue.
- About data modelling Erik Wilde added that Google is very opportunistic but also very
much driven by adoption and utility. Bert Van Nuffelen added that it is not build on firm
semantics, that a government is bound by that but that, however, mapping the firm gov
semantics to it is needed.
- Erik Wilde mentioned RDF and Bert Van Nuffelen noticed that it is not because you have a
RDF model it is trustworthy, but that it makes linking though easier.
Security of APIs
This topic collected lots of comment, as it is one of the most important challenges when
implementing APIs.
- Erik Wilde informed that Erik Wittern is working on interesting security analyses for
GraphQL, but all of this is far some simple or easy.
- Roberto Polli posed the question about if encrypted content-coding is a suitable solution
for those issues. He would have preserved the schema and used a protocol feature to solve
it transparently. He informed that iirc Darrel Miller even proposed JOSE as a content-
coding. To answer the question of Roberto:
o Alan Glickenhouse suggested to check an online document of the IBM community
on the topic
(https://community.ibm.com/community/user/imwuc/viewdocument/principles-
for-api-security-white)
o Alexander Hoose answered that they are looking into that for procedure
oriented APIs for sending in application form data to government. Therefore, they
want to mandate to encrypt the formdata and encode it into a JWE Token before
sending the data to the API.
o Roberto Polli clarified that they mentioned JWE in their guidelines and that the
problem is that JWT is easily to implement in insecure way
(https://datatracker.ietf.org/doc/rfc8725/ ), e.g. JWE will not be implemented in
google/tink library(https://github.com/google/tink/issues/342#issuecomment-
658450381). More investigation should be done in something similar to
(https://tools.ietf.org/html/rfc8188) and reach out infosec communities in IETF
Useful links to policy context interesting related events
- Policy context
o The Open Data directive (https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=uriserv:OJ.L_.2019.172.01.0056.01.ENG)
o The Communication on An European Data strategy
(https://ec.europa.eu/info/strategy/priorities-2019-2024/europe-fit-digital-
age/european-data-strategy )
- Eventso Semic 2020 (https://joinup.ec.europa.eu/collection/semantic-interoperability-
community-semic/semic-2020-speakers)
- Data modelling (https://smartdatamodels.org/index.php/about/ )
- Web of things (https://www.w3.org/WoT/ )
- ISA2 programme awork bout semantics and common vocabularies
(https://ec.europa.eu/isa2/solutions/core-vocabularies_en ). Bert Van Nuffelen highlighted
that the ISA core vocabularies are indeed a good building block, but of-course they have
limits. E.g. typically the shared values are flattened to ensure that everyone can express
somehow the related information. When doing actually management on this data, more
structure is usually required. E.g. addresses. In the core vocabularies these are flat string
properties, whereas in INSPIRE this is a structured model (with firmer identities). Both have
use cases and are closely related, but are not the same.
Requests from the audience
- Alexander Hoose is interested in having further talk on the topic about very strict
specifications in the API specifications and also validate all tokensYou can also read
