API Gateway vs. Service Mesh

 Service Mesh "How is it different from an API gateway?" It's a good question. The overlap between API gateway and service mesh patterns is significant. They can both handle service discovery, request routing, authentication, rate limiting, and monitoring, but there are differences in architectures and intentions. A service mesh's primary purpose is to manage internal service-to-service communication, while an API Gateway is primarily meant for external client-to-service communication.

Do You Need Both?

You may be wondering if you need both an API gateway and a service mesh.  Today you probably do, but as service mesh evolves, we believe it will incorporate much of what you get from an API gateway today.

The main purpose of an API gateway is to accept traffic from outside your network and distribute it internally. The main purpose of a service mesh is to route and manage traffic within your network. A service mesh can work with an API gateway to efficiently accept external traffic then effectively route that traffic once it's in your network. The combination of these technologies can be a powerful way to ensure application uptime and resiliency while ensuring your applications are easily consumable.

In a deployment with an API gateway and a service mesh, incoming traffic from outside the cluster would first be routed through the API gateway, then into the mesh. The API gateway could handle authentication, edge routing and other edge functions, while the service mesh provides fine-grained observability of and control of your architecture.

The interesting thing to note is that service mesh technologies are quickly evolving and are starting to take on some of the functions of an API gateway. A great example is the introduction of the Istio v1alpha3 routing API which is available in Aspen Mesh 1.0. Prior to this, Istio had used Kubernetes ingress control which is pretty basic so it made sense to use an API gateway for better functionality. But, the increased functionality introduced by the v1alpha3 API has made it easier to manage large applications and to work with protocols other than HTTP, which was previously something an API gateway was needed to do effectively.

What The Future Holds

The v1alpha3 API provides a good example of how a service mesh is reducing the need for API gateway capabilities. As the cloud-native space evolves and more organizations move to using Docker and Kubernetes to manage their microservice architectures, it seems highly likely that service mesh and API gateway functionality will merge. In the next few years, we believe that standalone API gateways will be used less and less as much of their functionality will be absorbed by service mesh.

References:

https://dzone.com/articles/api-gateway-vs-service-mesh

https://konghq.com/blog/the-difference-between-api-gateways-and-service-mesh/

https://medium.com/microservices-in-practice/service-mesh-vs-api-gateway-a6d814b9bf56

Choosing between the tools is at https://medium.com/@mahesh.mahadevan/my-experiences-with-api-gateways-8a93ad17c4c4

REST API Guidelines

REST API Guidelines

The below link has comprehensive guidelines Microsoft REST APIs should follow so RESTful interfaces are developed consistently. And these can be followed by API developers and Architects at other companies too.

Refer:

https://github.com/microsoft/api-guidelines/blob/vNext/Guidelines.md

Why Digital is The Future! How to have a successful journey...

Why Digital is The Future! How to have a successful journey...

"Digital" and "digitization" or "online" has become a very common word in the recent past due to the C-19 pandemic situation, and yet, the digital transformation has been there in every industry more than two decades or so to create value-driven business opportunities for the organizations in their served customer segments with the right value proposition to fulfill their internal and external customers with amazing customer experiences (CX).

Some basics to know- The average IT spend across all industries was around 8 percent of their revenues as a magic number and yet it depends on many factors like industry, region, and age of the company, etc. Now, think about organizations spending between 4 to 24 percent based on the industry for their IT or digital goals, wherein the staff of IT and CIO or CDO would be bringing seamless and easy to use tools and solutions for the various businesses including sales and markets and other business functions in the company (plus external customers too). 

On a side note, usually, companies spend roughly around 10 percent as a magic number on sales and marketing, with a disclaimer that it depends on the size of the company and which sector it belongs to, etc.

With that, it's very imperative that you as a CTO or CDO or CIO or Digital IT Head need to have a very clear plan to create a successful journey towards it with the available budgets in place... 

Some of the key caveats for the success of any organization to become truly digital are:

• Make sure to address the five domains of the digital strategy - customers, competition, data, innovation, and value
• Create a comprehensive digital platform with digital products, and not siloed or isolated apps or customizations
• Start with small to focus more on the CX, and then slowly evolve and improvise your design
• How fast you pivot to digital is the clear caveat to bring more efficiencies and value to your businesses and their respective functions
• Have a clear business architecture and enterprise architecture in place whenever your organization do any M&As and adapt as quickly as possible so that all the respective fulfillment happens seamlessly for the existing (for example, legacy apps) and newly acquired customers
• Application and/or software modernization with the latest emerging technologies like microservices, cloud, machine learning, artificial intelligence, monitoring, cybersecurity, etc
• IT architecture is very important to any organization, and yet, that needs to be refined every now and then to meet your business outcomes and demands
• Having an application rationalization chart to know the right sunset and sunrise digital products based on your organization priorities so that it is easier to maintain and innovate faster while the systems and applications are co-existing with newer digital products; a clear winner to achieve improvements in business operations
• Always, treat your data as the biggest asset to your organization to do any sort of transformation and decision making
• Make sure to have highly available, reliable, and accurate data APIs/Services for the key source of truth from the various data stores, these can be called as DaaS (Data as a Service) to ease accessibility and minimize the data issues
• A robust operational backbone is necessary but that certainly is not sufficient for an organization's digital future, an organization needs to adapt to the newer technologies and have to innovate faster to create newer digital products or try to leverage existing newer COTS (Commercial off-the-shelf) tools to create newer digital solutions to continue delight and address all of your customer behaviors
• Always, have shared data analytics or insights from the data that you have collected from customers
• Re-use, Re-design, and Re-create wherever and whenever required to save time and money
• Have intelligent automation in place for automating some/many of the most used business processes
• Promote continuous delivery and/or integration with DevOps and SRE (Site reliability engineering) principles, so that feedback loops/cycles are faster and teams can implement and address them in a nimble fashion
• Promote co-creations and co-ownerships with the right internal and external accountability frameworks with faster pace decision making and approvals etc
• "digital" is everyone's job to become truly a digital organization. For example, to have the right business owners or product owners and then create more collaborative and agile ways of working together to reduce risks and dependencies
• Try to create a mindset where the customer is more important than anything, and providing seamless customer experience matters a lot
• Treat your tech debt as an innovation aspect of the digital journey, and have the organizational culture as design thinking instead of leaner thinking to come up with innovative digital solutions
• Having a digital product mindset instead of a project mindset
• Also, an organization can focus mostly on their core foundations and innovation only by having a great powerful external partner and/or developer community who gives you early feedbacks and promote your organization with ever scaling digital platforms
• Have resiliency aspects in your planned mission-critical releases, plan very well for the unplanned occurrences, and have architecture design for the next billion users

References/books:

Enterprise Architecture As Strategy: Creating a Foundation for Business Execution

Designed for Digital: How to Architect Your Business for Sustained Success (Management on the Cutting Edge)

Why Digital Transformations Fail: The Surprising Disciplines of How to Take Off and Stay Ahead

The Lean Product Playbook: How to Innovate with Minimum Viable Products and Rapid Customer Feedback

https://www.amazon.com/Enterprise-Architecture-Strategy-Foundation-Execution/dp/1591398398

https://www.bmc.com/blogs/digital-transformation-books/

https://www.cnbc.com/2019/04/08/4-trillion-in-tech-spending-in-2019-heres-where-the-money-is-going.html

https://vtldesign.com/digital-marketing/content-marketing-strategy/percent-of-revenue-spent-on-marketing-sales/

https://www.flexera.com/blog/elo/2020-state-of-tech-spend-it-spending-benchmarks-and-trends/

https://www.flexera.com/blog/industry-trends/it-spending-by-industry/

https://blog.techvera.com/company-it-spend

Database migrations aren't tough; use the right tools

Database migrations are not tough and it's all about how you use the right tools to overcome your most common issues for your on-going database development changes.

Most famous open source tools are:

1. https://www.liquibase.org/
2. https://flywaydb.org/
3. Many more...

So, what to choose and when to use liquibase vs. flyway

• liquibase and flyway has more or less same features, except the flyway uses the SQL as a script whereas liquibase supports SQL, YAML, XML, JSON
• Easier setup and configurations then, flyway; liquibase also easy setup though
• Both flyway and liquibase have tooling and integrations support with your CI/CD tech stack
• Little more configurations with more features you need then, go for liquibase 
• liquibase has a lot of databases supported
• liquibase has database versions controlling
• liquibase has undoing the migrations, and doing the diff etc whereas, it's premium in flyway
• smaller teams and many projects then, go flyway
• bigger teams with many projects then go for liquibase as you can have rollback and diff features as all varieties of scripts can be supported

References:

https://medium.com/@ruxijitianu/database-version-control-liquibase-versus-flyway-9872d43ee5a4

https://github.com/flyway/flyway

https://github.com/liquibase/liquibase

Kong API Gateway : Best Practices

Let me have some reference notes here in this blog for the best practices that can be followed with respect to the Kong API gateway and their Architecture Patterns.

Background

Kong’s overall infrastructure is composed of three main parts: NGINX provides protocol implementations and worker process management, OpenResty provides Lua integration and hooks into NGINX’s request processing phases, and Kong itself utilizes those hooks to route and transform requests.

Kong Architecture Patterns

• Simple Kong between client and upstream as 'a proxy'
• Kong and APIs behind a firewall for more control over the APIs
• Separate Kong Admin node for better security 
• Multi-datacenter with Cassandra for high availability to achieve higher throughput

References:

https://docs.konghq.com/enterprise/0.32-x/kong-architecture-patterns/

https://konghq.com/blog/microservices-high-availability/

https://docs.konghq.com/enterprise/0.32-x/kong-architecture-overview/

https://www.codit.eu/blog/kong-the-upcoming-king-of-the-api-jungle/

https://medium.com/swlh/kong-api-gateway-zero-to-production-5b8431495ee

API Gateway : Architecture Patterns

The simple definition of API Gateway Architecture Pattern is: API gateway Implement a service which is the entry point into the microservices-based application from external API clients. 

As you may know, there are numerous drawbacks with services accessing services directly. It’s often impractical for a client to perform API composition over the Internet. The lack of encapsulation makes it difficult for developers to change service decomposition and APIs. Services sometimes use communication protocols that aren’t suitable outside of the firewall. Consequently, a much better approach is to use what’s known as an API gateway.

An API gateway is a service that is the entry point into the application from the outside world. It’s responsible for request routing, API composition, and other functions, such as authentication. Let’s take a look at the API gateway pattern.

Overview of the API gateway pattern

The drawbacks of clients making multiple requests in order to display information to the user are well known (I describe them in my book!). A much better approach is for a client to make a single request to what’s known as an API gateway. An API gateway is a service that is the single entry-point for API requests into an application from outside the firewall. It’s similar to the Facade pattern from object-oriented design. Like a facade, an API gateway encapsulates the application’s internal architecture and provides an API to its clients. It might also have other responsibilities, such as authentication, monitoring, and rate-limiting. Figure 1 shows the relationship between the clients, the API gateway, and the services.

---

Figure 1. The API gateway is the single entry-point into the application for API calls from outside the firewall

---

The API gateway is responsible for request routing, API composition, and protocol translation. All API requests from external clients first go to the API gateway. The API gateway routes some requests to the appropriate service. The gateway handles other requests using the API Composition pattern and invoking multiple services and aggregating the results. It might also translate between client-friendly protocols such as HTTP and WebSockets and client-unfriendly protocols that are used by the services.

Request routing

One of the key functions of an API gateway is request routing. An API gateway implements some API operations by routing requests to the corresponding service. When it receives a request, the API gateway consults a routing map that specifies which service to route the request to. A routing map might, for example, map an HTTP method and path to the HTTP URL of the service. This function is identical to the reverse proxying features provided by web servers such as NGINX.

API Composition

An API gateway typically does more than reverse proxying. It might also implement some API operations using API composition. In my book, I use an example application called FTGO to teach how to move from a monolithic architecture to microservices. The FTGO API gateway, for example, implements the Get Order Details API operation using API composition. As figure 2 shows, the mobile application makes one request to the API gateway, which fetches the order details from multiple services.

---

Figure 2. An API gateway often does API Composition, which enables a client such as mobile device to efficiently retrieve data using a single API request.

---

The FTGO API gateway provides a coarse-grained API that enables mobile clients to retrieve the data they need with a single request. For example, the mobile client makes a single getOrderDetails() request to the API gateway.

Protocol translation

An API gateway might also perform protocol translation. The API gateway might provide a RESTful API to external clients even though the application services use a mixture of protocols internally include REST and gRPC. When needed, the implementation of some API operations translate between the RESTful external API and the internal gRPC-based APIs.

The API gateway provides each client with client-specific API

An API gateway could provide a single one-size-fits-all (OSFA) API. The problem with a single API is that different clients often have different requirements. For instance, a third-party application might require the Get Order Details API operation to return the complete Order details, while a mobile client only needs a subset of the data. One way to solve this problem is to give clients the option of specifying in a request which fields and related objects the server should return. This approach is adequate for a public API that must serve a broad range of third-party applications, but it often doesn’t give clients the control they need.

A better approach is for the API gateway to provide each client with its own API. For example, the FTGO API gateway can implement provide the FTGO mobile client with an API that is specifically designed to meet its requirements. It might even have different APIs for the Android and iPhone mobile applications. The API gateway also implements a public API for third-party developers to use.

Implementing edge functions

Although an API gateway’s primary responsibilities are API routing and composition, it may also implement what are known as edge functions. An edge function is, as the name suggests, a request processing function that is implemented at the edge of an application.

Examples of edge functions that an application might implement include:

• authentication – verifying the identity of the client making the request
• authorization – verifying that the client is authorized to perform that particular operation
• rate-limiting – limiting how many requests per second are allowed from either a specific client and/or from all clients
• caching – cache responses to reduce the number of requests made to the services
• metrics collection – collect metrics on API usage for billing analytics purposes
• request logging – log requests

Three different places in your application allow you to implement these edge functions.

The first option is to implement these edge functions in the backend services. This might make sense for some functions such as caching, metrics collection, and possibly authorization, but it’s generally more secure if the application authenticates requests on the edge before they reach the services.

The second option is to implement these edge functions in an edge service which is upstream from the API gateway. The edge service is the first point of contact for an external client. It authenticates the request and performs other edge processing before passing it to the API gateway.

An important benefit of using a dedicated edge service is that it separates concerns. The API gateway focuses on API routing and composition. Another benefit is that it centralizes responsibility for critical edge functions such as authentication. This particularly valuable when, as I describe below, an application has multiple API gateways that are possibly written using a variety of languages and frameworks. The drawback of this approach is that increases network latency because of the extra hop. It also adds to the complexity of the application.

As a result, it’s often convenient to use the third option and implement these edge functions, like authorization, in the API gateway itself. One less network hop exists, improving latency. Fewer moving parts reduces complexity.

API gateway architecture

An API gateway has a layered, modular architecture. Its architecture, which is shown in figure 3, consists of two layers, the API layer and a common layer. The API layer consists of one or more independent API modules. Each API module implements an API for a particular client. The common layer implements shared functionality including edge functions such as authentication.

---

Figure 3. An API gateway has a layered modular architecture. The API for each client is implemented by a separate module. The common layer implements functionality which is common to all APIs such as authentication

---

In this example, the API gateway has three API modules:

• Mobile API, which implements the API for the FTGO mobile client
• Browser API, which implements the API to the JavaScript application running in the browser
• Public API, which implements the API for third-party developers.

An API module implements each API operation in one of two ways. Some API operations map directly to single service API operation. An API module implements these operation by routing requests to the corresponding service API operation. It might implement these API operations using a generic routing module that reads a configuration file describing the routing rules.

An API module implements other, more complex API operations using API composition. The implementation of this API operation consist of custom code. Each API operation implementation handles requests by invoking multiple services and combining the results.

API gateway ownership model

An important question that you must answer is who is responsible for the development of the API gateway and its operation? One option is for a separate team to be responsible for the API gateway. The drawback of this option is that it’s similar to SOA, where an Enterprise Service Bus (ESB) team was responsible for all ESB development. If a developer working on the mobile application needs access to a particular service, they must submit a request to the API gateway team and wait for them to expose the API. This kind of centralized bottleneck in the organization is counter to the philosophy of the microservice architecture, which promotes loosely-coupled autonomous teams.

A better approach, which has been promoted by Netflix, is for the client teams—the mobile, web, and public API teams—to own the API module that exposes their API. An API gateway team is responsible for developing the Common module and for the operational aspects of the gateway. This ownership model, which is shown in figure 4, gives the teams control over their APIs.

---

Figure 4. A client team owns their API module. As they change the client, they can change the API module and not ask the API Gateway team to make the changes.

---

When a team needs to change their API, they check in the changes to the source repository for the API gateway. In order to work well, the API gateway’s deployment pipeline must be fully automated. Otherwise, the client teams are often blocked waiting for the API gateway team to deploy the new version.

Using the Backends for front ends pattern

One concern with an API gateway is that responsibility for it is blurred. Multiple teams contribute to the same code base. An API gateway team is responsible for its operation. Although not as bad as an SOA ESB, this blurring of responsibilities is counter to the microservice architecture philosophy of “if you build, it you own it.”

The solution is to have an API gateway for each client, the Backends for front ends (BFF) pattern, which was pioneered by Phil Calçado (http://philcalcado.com/) and his colleagues at SoundCloud. As figure 5 shows, each API module becomes its own standalone API gateway which is developed and operated by a single client team.

---

Figure 5. The Backend for front end pattern defines a separate API gateway for each client. Each client team owns their API gateway. An API gateway team owns the common layer.

 

Pattern: Backends for front ends

Implement a separate API gateway  for each type of client. See:http://microservices.io/patterns/apigateway.html.

 

The Public API team owns and operates their API gateway, the mobile team owns and operates theirs, and so on. In theory, different API gateways could be developed using different technology stacks, but this risks duplicating code for common functionality such as the code that implements edge functions. Ideally, all API gateways use the same technology stack. The common functionality is a shared library implemented by the API gateway team.

As well as clearly defining responsibilities, the BFF pattern has other benefits. The API modules are isolated from one another, which improves reliability. One misbehaving API can’t readily impact other APIs. The BFF pattern also improves observability because different API modules are different processes. Another benefit of the BFF pattern is that each API is independently scalable. And the BFF pattern reduces startup time, because each API gateway is a smaller, simpler application.

Benefits and drawbacks of an API gateway

As you might expect, the API gateway pattern has both benefits and drawbacks.

Benefits of an API gateway

A major benefit of using an API gateway is that it encapsulates the internal structure of the application. Rather than having to invoke specific services, clients talk to the gateway. The API gateway provides each client with a client-specific API. This reduces the number of round trips between the client and application. It also simplifies the client code.

Drawbacks of an API gateway

The API gateway pattern also has some drawbacks. It’s yet another highly available component that must be developed, deployed, and managed. This creates the risk that the API gateway becomes a development bottleneck. Developers must update the API gateway in order to expose their services’ API. It’s important that the process for updating the API gateway is as lightweight as possible. Otherwise, developers are forced to wait in line in order to update the gateway.

Despite these drawbacks, for most real-world applications it makes sense to use an API gateway. If necessary, you can use the Backends for front ends patterns to enable the teams to develop and deploy their APIs independently.

Netflix as an example of an API gateway

A great example of an API gateway is the Netflix API. The Netflix streaming service is available on hundreds of different kinds of devices, including televisions, Blu-ray players, smart phones, and so on. Initially, Netflix attempted to have a one-size-fits-all  style API for their streaming service (http://www.programmableweb.com/news/why-rest-keeps-me-night/2012/05/15). Unfortunately, they soon discovered that it didn’t work well because of the diverse range of devices and their different needs. Today, they use an API gateway that implements a separate API for each device. The client device team develops and owns the API implementation.

In the first version of the Netflix API gateway, each client team implemented their API using Groovy scripts that perform routing and API composition. Each script invoked one or more service APIs using Java client libraries provided by the service teams. On the one hand, this works well and client developers have written over thousands of scripts. The Netflix API gateway handles billions of requests per day; on average, each API calls fans out to six to seven backend services. Netflix has found this monolithic architecture to be somewhat cumbersome.

As a result, Netflix is now moving to an API gateway architecture that is similar to the Backends for front-end patterns. In this new architecture, client teams write API modules using NodeJS. Each API module runs its own Docker container. The scripts don’t invoke the services directly. Instead, they invoke a second “API gateway,” which exposes the service APIs using Netflix Falcor. Netflix Falcor is an API technology that does declarative, dynamic API composition, and enables a client to invoke multiple services using a single request. This new architecture has a number of benefits. The API modules are isolated from one another, which improves reliability and observability. Also, the client API module is independently scalable.

References/Sources:

http://microservices.io/patterns/apigateway.html 

https://freecontent.manning.com/the-api-gateway-pattern

https://tsh.io/blog/design-patterns-in-microservices-api-gateway-bff-and-more/

https://medium.com/dev-genius/microservices-design-api-gateway-pattern-980e8d02bdd5

https://www.nginx.com/blog/building-microservices-using-an-api-gateway/

Comprehensive info along with comparative analysis and rankings on all Databases

Knowledge Base of Relational and NoSQL Database Management Systems

DB-Engines is an initiative to collect and present information on database management systems (DBMS). In addition to established relational DBMS, systems and concepts of the growing NoSQL area are emphasized.

https://db-engines.com/en/ranking

All API Development Tools in one place!

All API Development Tools in one place

A collection of useful resources for building RESTful HTTP+JSON APIs.

https://github.com/yosriady/api-development-tools

IoT : The Internet of Things

 The Internet of Things is an emerging topic of technical, social, and economic significance. Consumer products, durable goods, cars and trucks, industrial and utility components, sensors, and other everyday objects are being combined with Internet connectivity and powerful data analytic capabilities that promise to transform the way we work, live, and play. Projections for the impact of IoT on the Internet and economy are impressive, with some anticipating as many as 100 billion connected IoT devices and a global economic impact of more than $11 trillion by 2025.

At the same time, however, the Internet of Things raises significant challenges that could stand in the way of realizing its potential benefits. Attention-grabbing headlines about the hacking of Internet-connected devices, surveillance concerns, and privacy fears already have captured public attention. Technical challenges remain and new policy, legal and development challenges are emerging.

This overview document is designed to help the Internet Society community navigate the dialogue surrounding the Internet of Things in light of the competing predictions about its promises and perils. The Internet of Things engages a broad set of ideas that are complex and intertwined from different perspectives. Key concepts that serve as a foundation for exploring the opportunities and challenges of IoT include:

• IoT Definitions: The term Internet of Things generally refers to scenarios where network connectivity and computing capability extends to objects, sensors and everyday items not normally considered computers, allowing these devices to generate, exchange and consume data with minimal human intervention. There is, however, no single, universal definition.
• Enabling Technologies: The concept of combining computers, sensors, and networks to monitor and control devices has existed for decades. The recent confluence of several technology market trends, however, is bringing the Internet of Things closer to widespread reality. These include Ubiquitous Connectivity, Widespread Adoption of IP-based Networking, Computing Economics, Miniaturization, Advances in Data Analytics, and the Rise of Cloud Computing.
• Connectivity Models: IoT implementations use different technical communications models, each with its own characteristics. Four common communications models described by the Internet Architecture Board include: Device-to-Device, Device-to-Cloud, Device-to-Gateway, and Back-End Data-Sharing. These models highlight the flexibility in the ways that IoT devices can connect and provide value to the user.
• Transformational Potential: If the projections and trends towards IoT become reality, it may force a shift in thinking about the implications and issues in a world where the most common interaction with the Internet comes from passive engagement with connected objects rather than active engagement with content. The potential realization of this outcome – a “hyperconnected world” — is testament to the general-purpose nature of the Internet architecture itself, which does not place inherent limitations on the applications or services that can make use of the technology.

Five key IoT issue areas are examined to explore some of the most pressing challenges and questions related to the technology. These include security; privacy; interoperability and standards; legal, regulatory, and rights; and emerging economies and development.

Security

While security considerations are not new in the context of information technology, the attributes of many IoT implementations present new and unique security challenges. Addressing these challenges and ensuring security in IoT products and services must be a fundamental priority.Users need to trust that IoT devices and related data services are secure from vulnerabilities, especially as this technology become more pervasive and integrated into our daily lives. Poorly secured IoT devices and services can serve as potential entry points for cyber attack and expose user data to theft by leaving data streams inadequately protected.

The interconnected nature of IoT devices means that every poorly secured device that is connected online potentially affects the security and resilience of the Internet globally. This challenge is amplified by other considerations like the mass-scale deployment of homogenous IoT devices, the ability of some devices to automatically connect to other devices, and the likelihood of fielding these devices in unsecure environments.

As a matter of principle, developers and users of IoT devices and systems have a collective obligation to ensure they do not expose users and the Internet itself to potential harm. Accordingly, a collaborative approach to security will be needed to develop effective and appropriate solutions to IoT security challenges that are well suited to the scale and complexity of the issues.

Privacy

The full potential of the Internet of Things depends on strategies that respect individual privacy choices across a broad spectrum of expectations. The data streams and user specificity afforded by IoT devices can unlock incredible and unique value to IoT users, but concerns about privacy and potential harms might hold back full adoption of the Internet of Things. This means that privacy rights and respect for user privacy expectations are integral to ensuring user trust and confidence in the Internet, connected devices, and related services.

Indeed, the Internet of Things is redefining the debate about privacy issues, as many implementations can dramatically change the ways personal data is collected, analyzed, used, and protected. For example, IoT amplifies concerns about the potential for increased surveillance and tracking, difficulty in being able to opt out of certain data collection, and the strength of aggregating IoT data streams to paint detailed digital portraits of users. While these are important challenges, they are not insurmountable. In order to realize the opportunities, strategies will need to be developed to respect individual privacy choices across a broad spectrum of expectations, while still fostering innovation in new technology and services.

Interoperability / Standards

A fragmented environment of proprietary IoT technical implementations will inhibit value for users and industry. While full interoperability across products and services is not always feasible or necessary, purchasers may be hesitant to buy IoT products and services if there is integration inflexibility, high ownership complexity, and concern over vendor lock-in.

In addition, poorly designed and configured IoT devices may have negative consequences for the networking resources they connect to and the broader Internet. Appropriate standards, reference models, and best practices also will help curb the proliferation of devices that may act in disrupted ways to the Internet. The use of generic, open, and widely available standards as technical building blocks for IoT devices and services (such as the Internet Protocol) will support greater user benefits, innovation, and economic opportunity.

Legal, Regulatory and Rights

The use of IoT devices raises many new regulatory and legal questions as well as amplifies existing legal issues around the Internet. The questions are wide in scope, and the rapid rate of change in IoT technology frequently outpaces the ability of the associated policy, legal, and regulatory structures to adapt.

One set of issues surrounds crossborder data flows, which occur when IoT devices collect data about people in one jurisdiction and transmit it to another jurisdiction with different data protection laws for processing. Further, data collected by IoT devices is sometimes susceptible to misuse, potentially causing discriminatory outcomes for some users. Other legal issues with IoT devices include the conflict between law enforcement surveillance and civil rights; data retention and destruction policies; and legal liability for unintended uses, security breaches or privacy lapses.

While the legal and regulatory challenges are broad and complex in scope, adopting the guiding Internet Society principles of promoting a user’s ability to connect, speak, innovate, share, choose, and trust are core considerations for evolving IoT laws and regulations that enable user rights.

Emerging Economy and Development Issues

The Internet of Things holds significant promise for delivering social and economic benefits to emerging and developing economies. This includes areas such as sustainable agriculture, water quality and use, healthcare, industrialization, and environmental management, among others. As such, IoT holds promise as a tool in achieving the United Nations Sustainable Development Goals.

The broad scope of IoT challenges will not be unique to industrialized countries. Developing regions also will need to respond to realize the potential benefits of IoT. In addition, the unique needs and challenges of implementation in less-developed regions will need to be addressed, including infrastructure readiness, market and investment incentives, technical skill requirements, and policy resources.

The Internet of Things is happening now. It promises to offer a revolutionary, fully connected “smart” world as the relationships between objects, their environment, and people become more tightly intertwined. Yet the issues and challenges associated with IoT need to be considered and addressed in order for the potential benefits for individuals, society, and the economy to be realized.

Ultimately, solutions for maximizing the benefits of the Internet of Things while minimizing the risks will not be found by engaging in a polarized debate that pits the promises of IoT against its possible perils. Rather, it will take informed engagement, dialogue, and collaboration across a range of stakeholders to plot the most effective ways forward.

Refer for more details: https://www.internetsociety.org/wp-content/uploads/2017/08/ISOC-IoT-Overview-20151221-en.pdf

We all know SOA, but what is ROA?

We all know SOA(that is Service Oriented Architecture), but what is ROA?

ROA stands for Resource-Oriented Architecture, and what are the Resources here? The resources are nothing but your RESTful services or interfaces. This would become a paradigm shift in the overall web services world very soon and every company would adapt to the ROA instead of SOA to gain more traction in their way of doing businesses and services.

Basics of  RESTful services or interfaces:

REST is an architecture style for designing networked applications. It stands for REpresentational State Transfer. 

What is not REST?

• A language
• A framework
• A brand
• Software that you can download and make it run.
• A standard (W3C hasn't accepted, so it continues as open style)

REST constraints

As REST creator, Roy Fielding describes in his doctoral dissertation, it was conceived as a whole sets of needs, shaped by the constraints of the environment in which this system was going to be implemented. Such constraints are:

• Client-Server
• Uniform Interface
• Stateless
• Cacheable
• Layered System
• Code on demand (optional)

Some definitions before

State

A state is that which the server application cares about to fulfill the request. This is the chunk of information that is sent by the server in every response.

Resource

A resource is data that defines a model representation. This might be the stored data in the database.

Note, that, a state is a piece of information that depends on the request and the client that is asking for; while the resource is information that is constant through all clients (of course until it needs to be updated).

Idempotence

In terms of computer science, this term has a different meaning. It describes an operation that will produce the same result whether it is being called once or many times. In other words, making multiple identical requests has the same effect as making a single one. Imagine a client asking a server to update the resource, setting its property to the value of 20. No matter how many times you ask this. You'll have the same effect on the database.

Safety

Speaking of RESTful approach, safety means the ability of a method to not change the state of a resource in the server. This means, all methods that are safety doesn't alter the information (database). Consequently, clients can make safe requests repeatedly without worry of side effects on the server.

Client-Server

The web operates in this way: where there is a web server delivering content, that is being interpreted by a browser or a mobile app. In this case, the server is usually the web application, and the client is the browser, or mobile app. In this scenario, the information flow happens in one way: The client does a request to the server, which after processing such request, delivers a response.

Uniform Interface

This constraint defines that the interface between any server, and all possibilities of clients, should be the same. This decouples the architecture between client and server, giving the ability to both to evolve separately.

Stateless

Here it comes what REST means. Representational State Transfer means that each request has to carry with the necessary state to handle the transaction. This releases the server from keeping a state of information between multiple requests. Are you familiar with the concept of SESSION in web development ? REST avoids the existence of it. And because of this, the server gets the ability to scale better.

Cacheable

This constraint says that each response must define implicitly or explicitly, whether it is a cacheable resource, or not. By telling a client that such content is cacheable, it optimizes the performance of communication, since the client will avoid unnecessary requests for static data.

Layered System

This says that from the server-side, scalability may be improved in such way that the client won't notice whether it is dealing in one server or an intermediary app. That let the server side infrastructure to escalate through load balancing, shared caches, and security middlewares.

Code on demand (optional)

This means that the server might also delivers logic to the client, such as javascript content that might be interpreted by the client. This is optional, and I think is nowadays not used, since the appearance of Single Page Apps.

2. REST How-to

Principles

Know the best RESTful practices from the community.

1. Use meaningful HTTP verbs.

You know them: GET, POST, PUT, DELETE. Such verbs already mean an intention, and the purpose of REST principles is to use this as an advantage to set clarity for t he API.

2. Sensible resource names.

In the old times of web development, a resource used to accept parameters from long query strings, such as /api?model=user&id=123. By following this principle, the URI (also called resource name) shoud be as well meaninful as the HTTP verb. So rewriting the previous URI to a meaningful one, would be /user/123.

3. XML and JSON

That's up to the necessities of the client. If you're going to provide a public API, having both is a good idea, since you have a broad scope of clients that can interact with your app. But we suggest to favor JSON over XML. This is because the simplicity of its nature as a data container. Also, since most of the clients are browsers, JSON is javascript syntax, which makes perfect to be parsed by the client side. If you're going to render XML, avoid namespaces, and think of XML as a data container, which is the purpose of the API.

4. Create Fine-Grained Resources

Instead of starting with complex services, focus on basic, simple resources, and provide CRUD operations on them. Additionally you will add complex services over that system.

5. Idempotence for PUT and DELETE

This means that by sending a request with the same state (data) to the server, you'll have the same effect on the domain/database.

6. Safety for HEAD, GET, OPTIONS and TRACE

These methods are defined to be safe, this means, to not alter the state in the server.

HTTP Verbs

Technically, there are 8 HTTP Methods. The most used in RESTful APIs are only 4 (GET, POST, PUT and DELETE). They cover CRUD operations on a resource.

1. GET
2. POST
3. PUT
4. DELETE
5. OPTIONS
6. HEAD
7. TRANCE
8. CONNECT

1. GET

GET is used to retrieve information from the server, specifically information of a resource. If the request goes well, it returns a 200 code. Otherwise, it might return an error code, like 404 or 400

Examples:

# Gets a list of all users
GET /api/users
# Gets details for user with id = 23
GET /api/users/23
# Gets a list of all orders belonging to user with id = 23
GET /api/users/23/orders

2. POST

POST is used for creating a resource. It accepts usually a JSON body in the request. This data is the state for the resource that is going to be created. If the execution goes well, it returns a status code 201, with a location header with the link to the newly created resource. The POST operation is not safe, since it affects the state of information. It is also not an idempotent operation, since every time a resource creation is requested, the id attribute will be auto-incremented, making the effect of this operation, unique every time.

Examples:

# Inserts a user
POST /api/users
# body
{
  name: "User"
}

3. PUT

PUT is used for updating a resource. It accepts usually a JSON body which contains the data of the resource that will be updated. On succesful execution, it returns a status code 200. PUT is not a safe operation, since it affects the state of the resource in the server information. However, PUT is encouraged to be an idempotent operation. This is, by knowing the id of the resource that will be updated, and having certain values that will be written, every operation will cause the same effect on the resource status in the server.

Examples:

# Updates a user
PUT /api/users/23
# body
{
  name: "User number 23"
}

4. DELETE

DELETE operation is used for deleting a resource identified by a URI. This is not a safe operation, since it affects the state of the resource in the server. If the operation goes well, it responds with a satus code of 200. DELETE operation is idempotent. This is, by calling this operation the first time, the resource will be deleted from the server information storage (database), and by calling the next times, the response should be the same: the resource is gone.

Examples:

# Deleting a user
DELETE /api/users/delete/23

Methods Cheatsheet

HTTP Method	Resource Operation	Example URI	Response code	Location Header	Safe	Idempotent
GET	Fetch	/api/users, /api/users/{id}	200	No	Yes	No
POST	Create	/api/users	201	Yes	No	No
PUT	Update	/api/users/{id}	200	No	No	Yes
DELETE	Delete	/api/users/{id}	200	No	No	Yes

Response Status Codes

We will cover the most common codes here. If you want to check the full list, you can go to MDN page.

• 2xx Success
• 3xx Redirect
• 4xx User error
• 5xx Server error

2xx Success codes

• 200 OK (default code)
• 201 Created (Response for POST requests)
• 202 Accepted (Response for DELETE requests)

4xxx Error codes

• 400 Bad request (generic)
• 401 Unauthorized
• 404 Not found (Bad URL, resource not found)
• 405 Method not allowed (wrong HTTP method)
• 409 Conflict

3. Examples

When you have your api in the same domain that your web app

// prefix of /api to encapsulate API into one set of controller/routes
GET /api/users
GET /api/users/{id}

//another controllers in the webapp, traditional HTML response
GET /users
GET /users/{id}

When you have a specific domain for API, but different versions ov if

// API v1
GET /v1/users
GET /v1/users/{id}

// API v2.3
GET /v2.3/users
GET /v2.3/users/{id}

One domain, simple URI

GET /users
GET /users/{id}

Nested resources

//users is the main resource
GET /users
GET /users/{id}

//contracts is a sub resource of users
GET /users/{id}/contracts
GET /users/{id}/contracts/{contract_id}

Valid Methods for URIs

GET /users
GET /users/{id}
POST /users
PUT /users/{id}
DELETE /users/{id}

Invalid Methods for URIs

// you cannot create by specifying a specific resource
POST /users/{id}

// You cannot update state for a URI that doesn't represent a specific resource
PUT /users

// You cannot delete a resource if you no provide its specific URI
DELETE /users