Last Update 8:44 PM February 28, 2021 (UTC)

Identity Blog Catcher

Brought to you by Identity Woman and Infominer.
Support this collaboration on Patreon!!!

Sunday, 28. February 2021

Ben Werdmüller

One thing I’m finding challenging about losing ...

One thing I’m finding challenging about losing weight is separating the health reasons to do it - blood pressure, fitness, longevity - from the unhealthy body image reasons. I blame so many things on my own unattractiveness, and there’s a lot to unpack there. I’m trying.

One thing I’m finding challenging about losing weight is separating the health reasons to do it - blood pressure, fitness, longevity - from the unhealthy body image reasons. I blame so many things on my own unattractiveness, and there’s a lot to unpack there. I’m trying.


Bill Wendel's Real Estate Cafe

23rd National Consumer Protection Week: Why is Real Estate still a Blindspot?

Welcome to the 23rd annual National Consumer Protection Week, 2/28 – 3/6/21. Regrettably, other than our own Tweets, real estate once again appears to be… The post 23rd National Consumer Protection Week: Why is Real Estate still a Blindspot? first appeared on Real Estate Cafe.

Welcome to the 23rd annual National Consumer Protection Week, 2/28 – 3/6/21. Regrettably, other than our own Tweets, real estate once again appears to be…

The post 23rd National Consumer Protection Week: Why is Real Estate still a Blindspot? first appeared on Real Estate Cafe.


Ben Werdmüller

Really enjoying Star Trek, excited about the ...

Really enjoying Star Trek, excited about the next series of Doctor Who ... my 8 year old self would be so proud of how far I’ve come

Really enjoying Star Trek, excited about the next series of Doctor Who ... my 8 year old self would be so proud of how far I’ve come


Simon Willison

Trying to end the pandemic a little earlier with VaccinateCA

This week I got involved with the VaccinateCA effort. We are trying to end the pandemic a little earlier, by building the most accurate database possible of vaccination locations and availability in California. VaccinateCA I’ve been following this project for a while through Twitter, mainly via Patrick McKenzie - here’s his tweet about the project from January 20th. https://t.co/JrD5mb4TA

This week I got involved with the VaccinateCA effort. We are trying to end the pandemic a little earlier, by building the most accurate database possible of vaccination locations and availability in California.

VaccinateCA

I’ve been following this project for a while through Twitter, mainly via Patrick McKenzie - here’s his tweet about the project from January 20th.

https://t.co/JrD5mb4TAN calls medical professionals daily to ask who they could vaccinate and how to get in line. We publish this, covering the entire state of California, to help more people get their vaccines faster. Please tell your friends and networks.

- Patrick McKenzie (@patio11) January 20, 2021

The core idea is one of those things that sounds obviously correct the moment you hear it. The Covid vaccination roll-out is decentralized and pretty chaotic. VaccinateCA figured out that the best way to figure out where the vaccine is available is to call the places that are distributing it - pharmacies, hospitals, clinics - as often as possible and ask if they have any in stock, who is eligible for the shot and how people can sign up for an appointment.

What We've Learned (So Far) by Patrick talks about lessons learned in the first 42 days of the project.

There are three public-facing components to VaccinateCA:

www.vaccinateca.com is a website to help you find available vaccines near you. help.vaccinateca is the web app used by volunteers who make calls - it provides a script and buttons to submit information gleaned from the call. If you’re interested in volunteering there’s information on the website. api.vaccinateca is the public API, which is documented here and is also used by the end-user facing website. It provides a full dump of collected location data, plus information on county policies and large-scale providers (pharmacy chains, health care providers).

The system currently mostly runs on Airtable, and takes advantage of pretty much every feature of that platform.

Why I got involved

Jesse Vincent convinced me to get involved. It turns out to be a perfect fit for both my interests and my skills and experience.

I’ve built crowdsourcing platforms before - for MP’s expense reports at the Guardian, and then for conference and event listings with our startup, Lanyrd.

VaccinateCA is a very data-heavy organization: the key goal is to build a comprehensive database of vaccine locations and availability. My background in data journalism and the last three years I’ve spent working on Datasette have given me a wealth of relevant experience here.

And finally… VaccinateCA are quickly running up against the limits of what you can sensibly do with Airtable - especially given Airtable’s hard limit at 100,000 records. They need to port critical tables to a custom PostgreSQL database, while maintaining as much as possible the agility that Airtable has enabled for them.

Django is a great fit for this kind of challenge, and I know quite a bit about both Django and using Django to quickly build robust, scalable and maintainable applications!

So I spent this week starting a Django replacement for the Airtable backend used by the volunteer calling application. I hope to get to feature parity (at least as an API backend that the application can write to) in the next few days, to demonstrate that a switch-over is both possible and a good idea.

What about Datasette?

On Monday I spun up a Datasette instance at vaccinateca.datasette.io (underlying repository) against data from the public VaccinateCA API. The map visualization of all of the locations instantly proved useful in helping spot locations that had incorrectly been located with latitudes and longitudes outside of California.

I hope to use Datasette for a variety of tasks like this, but it shouldn’t be the core of the solution. VaccinateCA is the perfect example of a problem that needs to be solved with Boring Technology - it needs to Just Work, and time that could be spent learning exciting new technologies needs to be spent building what’s needed as quickly, robustly and risk-free as possible.

That said, I’m already starting to experiment with the new JSONField introduced in Django 3.1 - I’m hoping that a few JSON columns can help compensate for the lack of flexibility compared to Airtable, which makes it ridiculously easy for anyone to add additional columns.

(To be fair JSONField has been a feature of the Django PostgreSQL Django extension since version 1.9 in 2015 so it’s just about made it into the boring technology bucket by now.)

Also this week

Working on VaccinateCA has given me a chance to use some of my tools in new and interesting ways, so I got to ship a bunch of small fixes, detailed in Releases this week below.

On Friday I gave a talk at Speakeasy JS, "the JavaScript meetup for 🥼 mad science, 🧙‍♂️ hacking, and 🧪 experiments" about why "SQL in your client-side JavaScript is a great idea". The video for that is on YouTube and I plan to provide a full write-up soon.

I also recorded a five minute lightning talk about Git Scraping for next week's NICAR 2021 data journalism conference.

I also made a few small cosmetic upgrades to the way tags are displayed on my blog - they now show with a rounded border and purple background, and include a count of items published with that tag. My tags page is one example of where I've now applied this style.

TIL this week Using sphinx.ext.extlinks for issue links Show the SQL schema for a PostgreSQL database Running tests against PostgreSQL in a service container Adding extra read-only information to a Django admin change page Granting a PostgreSQL user read-only access to some tables Releases this week flatten-single-item-arrays: 0.1 - 2021-02-25
Given a JSON list of objects, flatten any keys which always contain single item arrays to just a single value datasette-auth-github: 0.13.1 - (25 releases total) - 2021-02-25
Datasette plugin that authenticates users against GitHub datasette-block: 0.1.1 - (2 releases total) - 2021-02-25
Block all access to specific path prefixes github-contents: 0.2 - 2021-02-24
Python class for reading and writing data to a GitHub repository csv-diff: 1.1 - (9 releases total) - 2021-02-23
Python CLI tool and library for diffing CSV and JSON files sqlite-transform: 0.4 - (5 releases total) - 2021-02-22
Tool for running transformations on columns in a SQLite database airtable-export: 0.5 - (7 releases total) - 2021-02-22
Export Airtable data to YAML, JSON or SQLite files on disk

Ben Werdmüller

Enjoyed a long hike today. It’s nice out there.

Saturday, 27. February 2021

John Philpin : Lifestream

“Some people can forcefully occupy the People’s House, whi

“Some people can forcefully occupy the People’s House, while others - those who are prevented from voting, who are aggressively policed, who are imprisoned at rates unmatched even by South Africa at the height of apartheid - are locked out.” The Resistance

“Some people can forcefully occupy the People’s House, while others - those who are prevented from voting, who are aggressively policed, who are imprisoned at rates unmatched even by South Africa at the height of apartheid - are locked out.”

The Resistance


Steinbeck’s Productive Inactivity. As one of my 4 favorit

Steinbeck’s Productive Inactivity. As one of my 4 favorite ‘John’s’ in the writing world … totally unsurprised that Cal Newport holds him up as an example of work efficacy. That said - 33 books in a lifetime is hardly taking it easy!

Steinbeck’s Productive Inactivity.

As one of my 4 favorite ‘John’s’ in the writing world … totally unsurprised that Cal Newport holds him up as an example of work efficacy. That said - 33 books in a lifetime is hardly taking it easy!


Erik Prince and the Failed Plot to Arm a Warlord in Libya

Erik Prince and the Failed Plot to Arm a Warlord in Libya … such a lovely chap.

Simon Willison

unasync

unasync Today I started wondering out loud if one could write code that takes an asyncio Python library and transforms it into the synchronous equivalent by using some regular expressions to strip out the "await ..." keywords and suchlike. Turns out that can indeed work, and Ratan Kulshreshtha built it! unasync uses the standard library tokenize module to run some transformations against an asyn

unasync

Today I started wondering out loud if one could write code that takes an asyncio Python library and transforms it into the synchronous equivalent by using some regular expressions to strip out the "await ..." keywords and suchlike. Turns out that can indeed work, and Ratan Kulshreshtha built it! unasync uses the standard library tokenize module to run some transformations against an async library and spit out the sync version automatically. I'm now considering using this for sqlite-utils.

Via @simonw


Ben Werdmüller

On writing in public

We get better at what we practice. Although I would never claim to be a perfect blogger, this kind of writing comes easily to me: I've been writing blog posts since 1998, and can track almost every career progression to something I wrote online. I love sharing my thoughts in this way, and I wish more people would do it. My feed reader is usually my first digital stop of the day. My personal pr

We get better at what we practice.

Although I would never claim to be a perfect blogger, this kind of writing comes easily to me: I've been writing blog posts since 1998, and can track almost every career progression to something I wrote online. I love sharing my thoughts in this way, and I wish more people would do it. My feed reader is usually my first digital stop of the day.

My personal project is to get better at writing fiction. Here, I'm far more awkward: I wrote widely when I was much younger, but I haven't been doing it for over a decade. I'm in awe of people like Eliot Peper and Cory Doctorow, who have been able to bridge a career in technology with careers as prolific novelists. And I have examples closer to home: my cousins Sarah Dessen and Jonathan Neale are both prolific authors. Sarah in particular has very kindly egged me on over the years, and I haven't quite followed through.

I think the first step is to get over the fear of starting something new - and move from the sinkhole of talking about writing to just doing it. Which is what I've been doing: over the last six months I've taken a handful of courses, which have forced me to produce work, and got to the final round of a fiction competition. These experiences have been positive: in particular, they've told me that I shouldn't throw in the towel, but also that I need to practice and improve. Being able to string a paragraph together is not the same skill as stringing a plot together.

One of the things I find addictive and compelling about blogging is you: it's a way to connect with people very immediately. These days it's rare for me to post something that doesn't generate a follow-on conversation. In contrast, writing creative work feels very isolated and time-delayed: you write something and iterate on it by yourself, maybe work with an editor, then submit it for publication or evaluation, which might come months later. That's the scary thing about it for me: unlike everything else I've ever done on the internet, the feedback loop is really offset and broken.

At the same time, your perceived worth as an author is still dependent on gatekeepers: while self-publishing has become more common, it remains important to be able to say "I've been published here, and here, and here." This is true throughout the creative world for forms that originated in legacy media: having a web series distributed by Netflix is markedly different to uploading it somewhere yourself, for example. Content forms that didn't originate in legacy media - TikTok clips, for example - have very different rules. But rules that have been established for decades or centuries are very hard to break through. Social norms are hard to change.

Ultimately, a reader doesn't want to have their time wasted, and I think the perception is that well-known publishers (or distributors) will protect their brands by standing for a certain level of quality. While self-published work can certainly be of the same or higher quality, it's a crapshoot. So finding a publisher for your work is important, and not a million miles away from finding a venture capitalist for your startup: you need to be able to find a coherent story for why your product will sell, and why they should bet on you. In the same way that many VCs only take warm introductions, many publishers will only accept work through an agent they already trust. Which, in turn, probably means establishing relationships.

For now, much of this is a problem for future me. Present me's problem is getting over the fear of sharing work, and finding ways to establish a productive feedback loop that will allow me to improve. (If you're a writer, I'd love to understand: how do you achieve this? Is it just that you're much braver than I am?)

I thought about creating a new community of beta readers, or establishing a new mailing list. I actually did create a mailing list some years ago for design fiction, but was never quite able to find a way to get it off the ground, perhaps because I'd defined its goals too formally, but perhaps also because I was scared that the work wasn't good enough to stand on its own in that way.

I think, instead, I'm going to use this space: I don't want to commit to a cadence or a particular style of work. But I want to have a place to put my experiments. It's categorically out of my comfort zone, and there's certainly a part of me that's worried I'll jettison all my subscribers. But this is a place I want to go, and I work better in the open, so that's how it's going to be.

Starting in March, expect regular fiction in this space. I find that idea really, really scary. But please do let me know what you think; you help me with your honesty.


Simon Willison

cosmopolitan libc

cosmopolitan libc "Cosmopolitan makes C a build-once run-anywhere language, similar to Java, except it doesn't require interpreters or virtual machines be installed beforehand. [...] Instead, it reconfigures stock GCC to output a POSIX-approved polyglot format that runs natively on Linux + Mac + Windows + FreeBSD + OpenBSD + BIOS with the best possible performance and the tiniest footprint imagi

cosmopolitan libc

"Cosmopolitan makes C a build-once run-anywhere language, similar to Java, except it doesn't require interpreters or virtual machines be installed beforehand. [...] Instead, it reconfigures stock GCC to output a POSIX-approved polyglot format that runs natively on Linux + Mac + Windows + FreeBSD + OpenBSD + BIOS with the best possible performance and the tiniest footprint imaginable." This is a spectacular piece of engineering.

Via Hacker News


Ben Werdmüller

Sonoma moon, from yesterday. The hills have mostly grown back at this point, although you can still see quite a bit of charring.


John Philpin : Lifestream

“Some people can forcefully occupy the People’s House, whi

“Some people can forcefully occupy the People’s House, while others—those who are prevented from voting, who are aggressively policed, who are imprisoned at rates unmatched even by South Africa at the height of apartheid—are locked out.” The Resistance

“Some people can forcefully occupy the People’s House, while others—those who are prevented from voting, who are aggressively policed, who are imprisoned at rates unmatched even by South Africa at the height of apartheid—are locked out.”

The Resistance


Password management app LastPass is tracking you on Android.

Password management app LastPass is tracking you on Android. Lovely!

Friday, 26. February 2021

Simon Willison

How to secure an Ubuntu server using Tailscale and UFW

How to secure an Ubuntu server using Tailscale and UFW This is the Tailscale tutorial I've always wanted: it explains in detail how you can run an Ubuntu server (from any cloud provider) such that only devices on your personal Tailscale network can access it.

How to secure an Ubuntu server using Tailscale and UFW

This is the Tailscale tutorial I've always wanted: it explains in detail how you can run an Ubuntu server (from any cloud provider) such that only devices on your personal Tailscale network can access it.


MyDigitalFootprint

Data For Better Decisions. Nature or Nurture?

“Every” management student has had to answer the exam question: “Leadership/ management: Nature or Nurture? - discuss” It is a paradox from either side of the argument, the logical conclusion always highlights the other has truth. The reality of leadership and management is that it is a complex adaptive system, and context enables your nature to emerge and nurturing to mature.  This is


“Every” management student has had to answer the exam question: “Leadership/ management: Nature or Nurture? - discuss” It is a paradox from either side of the argument, the logical conclusion always highlights the other has truth. The reality of leadership and management is that it is a complex adaptive system, and context enables your nature to emerge and nurturing to mature.  This is important because we also know there is a link between strategy, styles (leadership) and business structures.  In this article, we will unpack how your “nature or nurture” thinking-structure, affects outcomes.  Your thinking-structure is also a complex adaptive system as your peers and customers thinking, your companies “culture of structure” thinking affect you. BUT have you considered how your data structure and your data philosophy will have a direct and significant impact on outcomes? 

I’ve known that my neurodiversity package (severe dyslexia, mild high functioning autism, ADHD) informs how I interrupt the world as my “biological cortex” and gut-brain axis structures process sensory data and memory uniquely. I cannot modify my mind or brain’s basic structure any more than I could change my fingerprint, core DNA or the colour of my eyes; however, I can play with my microbiome. It’s an essential part of what makes me, me. My chemical and biological structures enable me to make sense of the world in my way.  Communication (language, words, music, song, dance, art, sound, movement, gesture) enables us to share the sense we create from patterns and align with others who approximate the same (tribe).  How we actually make sense (learn) is intensely debated, with one camp believing that language is our sense maker, assuming that we might observe patterns but cannot understand it without language? Other than that, we make sense and then create a language to communicate the insight we have gained.  Irrespective, language allows us to both structure and navigate our space and share the journey.  

Is how we structure or frame something nature or nurture?

Why does this question matter? We all read, speak and write differently, we all understand differently, but we use questions to clarify understanding, check meaning and create common interruption.  How we individually structure meaning is determined from the perspective we have been given (nature), from what we have been taught (nurture) and what we align to (bias).  Our structure is an ontology*. Imagine putting one person from each of our worlds religions or faith groups into a room, but assume no-one can speak the same language.  How and what would they agree or disagree about as there is no common structure (ontology)

* An ontology is “the set of things whose existence is acknowledged by a particular theory or system of thought.”  (The Oxford Companion to Philosophy) 

By way of example, the word “Evil” creates meaning for you as soon as you read it. Without a doubt, the nature of evil is a complex and nuanced area too often glossed over in a rush to present or evaluate the defences and theodicies. Let’s unpack the word using the super book “Making Evil” by Dr Julia Shaw.   Evil is an unavoidable part of life that we all encounter as we all suffer in one sense or another, but what makes something evil is a matter of framing/ structure/ ontology. “Natural evil” is the pain and suffering that arises from the natural world’s functioning or malfunctioning. “Moral evil” is the pain and suffering that results from conscious human action or inaction. It is evil where a person or people are to blame for the suffering that occurs; a crucial point is the blameworthiness of the person at fault. Moral evil, at its heart, results from the free choice of a moral agent. If we just look at the consequences, it is not always possible to tell whether moral evil has taken place or not; we have many mitigations. Therefore a level of moral evil can be found in the degree of intention and consequence.  However, if we compare death rates for natural evil (suffering) and moral evil at an extreme people killing people, the latter is a rounding error in the form of suffering in the world. The point is that by framing something, I can create a structure for understanding. Critically our structures frame our understanding.  

Critically our structures frame our understanding.  

Structures are ontologies which are philosphies. 

To explore that our structures frame our understanding, what ontology makes us human? When we look at the different view below, we can view humans in many different ways. Note: I have deliberately ignored the classical all living things ontology structure (insects, birds, fish, mammal, reptiles, plant).  The point is that your framing or how you structure something at the start leads to a guided conclusion. 

Our framing or how we structure something at the start leads to a guided conclusion.

Pick a different structure, and you get a different answer; the ontology creates a natural conclusion.  It is likely that if you pick a philosophy/ ontology/ structure, you can only get what that framing will shine a light on or enable.

It is likely that if you pick a philosphies/ ontology/ structure, you can only get what that structure will shine a light on or enable.

This matters because all data has structure!

I explore continually the future of the digital business, which are underpinned by data, privacy, consent and identity.  Data is Data (it is not oil or sunshine).  What is The Purpose of your Data? Quantum (Data) Risk. Does data create a choice? Data and KPI’s. Wisdom is just more data. Data can create what does not exist. Data is not Memory.

I am asking these questions of directors, boards, senior leadership teams and data/ data managers. Directors are accountable in law for ensuring no discrimination and health and safety, but how can we know what we know if we don’t know the structure or framing of the data that gave us the result.  If we assume - that is a risk.

Do you have a data philosophy, and what is it?  

What is the structure of your data by silo? Is there a single top-level ontology?  

Do you know the structure/ ontologies of data for your ecosystem? 

What is the attestation and rights of the data in our data lake? How do we check if we are using data for a different purpose than intended?

How would you detect the consequences in your decision making by the aggregation of data with different ontologies? 

The Directors are accountable in law for discrimination, health and safety, and decision making (S.172 companies act), but how can we know what we know if we don’t know or understand the structure/ ontology and its limits.  We can now longer assume, as it is a known risk.

For most, this is already too much detail and in the weeds!  If you want to go deeper, this is a fantastic paper. A survey of Top-Level Ontologies to inform the ontological choices for a Foundation Data Model

 

Summary 

We want to use data to make better decisions and create new value.  However, we need to recognise that our data has a structure (ontology). Our data’s very structure (designed or otherwise) creates bias, prevents certain outcomes from being created, and creates others. The reality is that our structures (ontologies) have already committed to the success or failure of your data strategy and business model.  

The reality is that our structures (ontologies) have already committed to the success or failure of your data strategy and business model.  

As a leader, have you asked what is the structure (ontology) of our data? Has your team informed you about the limitations of your data structure/ ontology on decision making? The CDO should be tasked with providing a map, matrix or translation table showing data sets linkage to ontologies and the implications. As we now depend on ecosystem data, do you know the ontologies of others in your ecosystem and how that affects your decision making capability?  Gaps in data sharing ontologies affect decisions and create Quantum Risk.  What assumptions do we make about data without knowing is essential for investment risk, as we are using public ESG data to make capital allocation decisions without knowing where the data came from, what ontology the data has, if the right analysis tools have been used. 


----

Implication 1.  Management and Leadership

The figure-of-8 diagram below shows two interconnected loops. The connection is the mindset of the leader. Outstanding leadership with an open mindset can choose which loop is best at this time.  Poor leadership will stick to the lower closed mindset loop.  The lower loop never starts with a different way of asking questions or solving problems.  Those in this self-confirming loop stick to the same biases, same decisions and same paradigms.  This creates the ideas of one culture and a fixed culture.   We have our way of doing it.  The approach is consistency; the methods are highly efficient and based on the $1bn profit last year, we know it works, and we should continue to do the same.  The reward mechanism, KPI and balanced scorecards are structured to keep the same highly efficient and effective thinking.  Is assumes that yesterday, today and tomorrow will create the same outcomes if we do it the same.  There is nothing wrong with this, and during times of stability, many have made vast fortunes with this approach.

Great leaders follow this loop when it is right but can also swop to the upper loop.  Such leaders sense a change. Such a “paradigm shift”, a concept identified by the American physicist and philosopher Thomas Kuhn, “is a fundamental change in the basic concepts and experimental practices of a scientific discipline”.  This shift means there is a new structure to understand (ontology). This paradigm shift has a new structure, which means that there is a need to determine the new culture to create value with a new structure.  Together a team will form an approach.  At this point, the team will question the shift and the assumptions that have led to change, setting a new mindset for the new order. 

Critically - understanding structure and ontology is crucial, and it is why I believe Data Philosophy, Data Ontology and better decisions based on data are current board issues. Still, they require new skills, are highly detailed, and often require a mind shift. 

Understanding structure and ontology is crucial for a data-driven digital board.



Implication 2.  AI and Automation

The Data Paradox.  How are you supposed to know how to create questions about something that you did not know we had to ask a question of?

Every child reads a book differently. A child learns to use questions to check and refine understanding. Every software engineer reads code differently. A software engineer is forced to check their understanding of the code and function by asking questions and by being asked questions. Whilst every AI will create sense from the data differently (ontology and code), right now, an AI cannot check its understanding of data by asking questions! Who could/would/ should the AI ask the clarification question of, and how do we check the person who answered is without bias? (Note I am not speaking about AQA).  

Sherlock Holmes in The Great Game says, “people do not like telling you things; they love to contradict you. Therefore if you want smart answers, do not ask a question. Instead, give a wrong answer or ask a question in such a way that it already contains the wrong information. It is highly likely that people will correct you”.  Do you do this to your data, or can you do this to your AI?

Today (Feb 2021), we cannot write an “algorithm” that detects if AI is going to create harm (evil). Partly because we cannot agree on “harm”, we cannot determine the unintended consequences, and we cannot bound harm for a person vs society.  

There is a drive towards automation for efficiency based on the analysis of data. As a Director, are you capable of asking the right questions to determine bias and prejudice created in the automated processes, the data structures, different ontologies, data attestation or bias in the processes?  Given Directors are accountable and responsible - indeed, this is a skill all board needs. Where is the audit and quota for these skills, can you prove it is available to the board? 




Phil Windley's Technometria

Building Decentralized Applications with Pico Networks

Summary: Picos make building decentralized applications easy. This blog post shows a heterarchical sensor network can built using picos. Picos are designed to form heterarchical, or peer-to-peer, networks by connecting directly with each other. Because picos use an actor model of distributed computation, parent-child relationships are very important. When a pico creates another pico,

Summary: Picos make building decentralized applications easy. This blog post shows a heterarchical sensor network can built using picos.

Picos are designed to form heterarchical, or peer-to-peer, networks by connecting directly with each other. Because picos use an actor model of distributed computation, parent-child relationships are very important. When a pico creates another pico, we say that it is the parent and the pico that got created is the child. The parent-child connection allows the creating pico to perform life-cycle management tasks on the newly minted pico such as installing rulesets or even deleting it. And the new pico can create children of its own, and so on.

Building a system of picos for a specific application requires programming them to perform the proper lifecycle management tasks to create the picos that model the application. Wrangler is a ruleset installed in every pico automatically that is the pico operating system. Wrangler provides rules and functions for performing these life-cycle management tasks.

Building a pico application can rarely rely on the hierarchical parent-child relationships that are created as picos are managed. Instead, picos create connections between picos by creating what are called subscriptions, providing bi-directional channels used for raising events to and making queries of the other pico.

This diagram shows a network of temperature sensors built using picos. In the diagram, black lines are parent-child relationships, while pink lines are peer-to-peer relationships between picos.

Temperature Sensor Network (click to enlarge)

There are two picos (one salmon and the other green) labeled a "Sensor Community". These are used for management of the temperature sensor picos (which are purple). These community picos are performing life-cycle management of the various sensor picos that are their children. They can be used to create new sensor picos and delete those no longer needed. Their programming determines what rulesets are installed in the sensor picos. Because of the rulesets installed, they control things like whether the sensor pico is active and how often if updates its temperature. These communities might represent different floors of a building or different departments on a large campus.

Despite the fact that there are two different communities of temperature sensors, the pink lines tell us that there is a network of connections that spans the hierarchical communities to create a single connected graph of sensors. In this case, the sensor picos are programmed to use a gossip protocol to share temperature information and threshold violations with each other. They use a CRDT to keep track of the number of threshold violations currently occuring in the network.

The community picos are not involved in the network interactions of the sensor picos. The sensor network operates independently of the community picos and does not rely on them for communication. Astute readers will note that both communities are both children of a "root" pico. That's an artifact of the way I built this, not a requirement. Every pico engine has a root pico that has no parent. These two communities could have been built on different engines and still created a sensor network that spanned multiple communities operating on multiple engines.

Building decentralized networks of picos is relatively easy because picos provide support for many of the difficult tasks. The actor model of picos makes them naturally concurrent without the need for locks. Picos have persistent, independent state so they do not depend on external data stores. Picos have a persistent identity—they exist with a single identity from the time of their creation until they are deleted. Picos are persistently available, always on and ready to receive messages. You can see more about the programming that goes into creating these systems in these lessons: Pico-Based Systems and Pico to Pico Subscriptions.

If you're intrigued and want to get started with picos, there's a Quickstart along with a series of lessons. If you want support, contact me and we'll get you added to the Picolabs Slack.

The pico engine is an open source project licensed under a liberal MIT license. You can see current issues for the pico engine here. Details about contributing are in the repository's README.

Tags: picos heterarchy p2p

Thursday, 25. February 2021

Bill Wendel's Real Estate Cafe

WAIT – Prices never go down in Cambridge, right?

The headline in Boston.com’s real estate section reads, Mass Home SALES stay hot in January, but buried in the story is a myth busting statistics:… The post WAIT - Prices never go down in Cambridge, right? first appeared on Real Estate Cafe.

The headline in Boston.com’s real estate section reads, Mass Home SALES stay hot in January, but buried in the story is a myth busting statistics:…

The post WAIT - Prices never go down in Cambridge, right? first appeared on Real Estate Cafe.


John Philpin : Lifestream

Today I learned about flushing my local cache - not the brow

Today I learned about flushing my local cache - not the browser, not the remote server, not the network. I know - right! Who knew! How could I have never hit this problem before? Still, all good - all working Many thanks to WPMUDEV support. They really are rather good!

Today I learned about flushing my local cache - not the browser, not the remote server, not the network.

I know - right! Who knew!

How could I have never hit this problem before? Still, all good - all working

Many thanks to WPMUDEV support.

They really are rather good!

Wednesday, 24. February 2021

Ben Werdmüller

I'm hiring engineers and product managers

Brass tacks: I have three roles at my company that I need to fill immediately. In each case, you'll be working with me directly. The first is a Senior Product Manager. I'm looking for someone who is comfortable leading sprints, writing stories, and working in an interdisciplinary way across teams, but particularly with the engineering and design teams. You've got to be hands on; you've got to h

Brass tacks: I have three roles at my company that I need to fill immediately. In each case, you'll be working with me directly.

The first is a Senior Product Manager. I'm looking for someone who is comfortable leading sprints, writing stories, and working in an interdisciplinary way across teams, but particularly with the engineering and design teams. You've got to be hands on; you've got to have direct experience as a Product Manager at a startup; it's preferable that you have Fintech experience. Apply here.

The second is a Senior Ruby on Rails Engineer. This is an open role on my team. You'll be helping to build back-end systems and integrations that will allow regular people to save for retirement using the tools, assets, and advice normally reserved for the wealthy. You've worked at a few startups at a high level and have been an engineering lead. Apply here.

The third is a Mid-Level Ruby on Rails Engineer. This is a similar role to the previous, but you don't need to have been an engineering lead. Apply here.

To be very clear: you will not be filtered based on where you went to school, your identity, or where you came from. I do care deeply about whether you're hands-on and empathetic, with a bias towards action. I'd love to work with you.

If you'd like to have a quick chat about these positions, I'd love to jump on the phone. Click here to set up a meeting.

In all cases, I'm able to hire anywhere in the United States. (Unfortunately if you're not in the US, I have to rule you out for now.)


This is the triumphant face of a person who walked all the way around the block today.


I’m looking forward to the moment when ...

I’m looking forward to the moment when we can all get vaccines.

I’m looking forward to the moment when we can all get vaccines.


John Philpin : Lifestream

Is Harvard denying Tenure to Cornel West over his views on P

Is Harvard denying Tenure to Cornel West over his views on Palestine? It’s hard to believe that this could be the case in this day and age … then again it’s equally easy to believe. #sad

Is Harvard denying Tenure to Cornel West over his views on Palestine?

It’s hard to believe that this could be the case in this day and age … then again it’s equally easy to believe.

#sad


The End of an Era So long Frys.

The End of an Era So long Frys.

The End of an Era

So long Frys.


Ben Werdmüller

The Green New Deal

I'm terrified for the future and not sure where to begin. I have a young, teenage cousin who has apparently been having panic attacks; not because of school or generalized anxiety, but because he has a real sense that the world will have disintegrated in his lifetime. The signs of climate change and our less than inadequate response to it are all around us. I'm not scared because it's happen

I'm terrified for the future and not sure where to begin.

I have a young, teenage cousin who has apparently been having panic attacks; not because of school or generalized anxiety, but because he has a real sense that the world will have disintegrated in his lifetime. The signs of climate change and our less than inadequate response to it are all around us.

I'm not scared because it's happening. We have to act swiftly, but I believe we can act. I'm scared because I don't think we will.

There are three distinct groups that I think are problematic. The first are the people directly making money from outdated technologies like fossil fuels, who will sabotage attempts to move us to more intelligent, renewable energy. The second are the people who refuse to believe that climate change exists, or who spread the lie that it's nothing to worry about. And the third are the people who are so addicted to capital that they can't imagine solving the problem outside of the markets.

I'm a proponent of the Green New Deal, which advocates a program of divestment from fossil fuels, government investments in renewable energy, and robust creation of public jobs to create sustainable infrastructure. Its comparison to the original New Deal is apt; the challenge we face is easily comparable to the devastating context of a world war.

It's a sensible and much-needed solution, but it's under attack from conservatives and centrists alike. Even Joe Biden said he didn't support it during the Presidential debates. The reason is simply that it upsets existing structures of power. A Green New Deal necessitates, in part, a redistribution of equity.

As Naomi Klein wrote recently, that doesn't go down well with free marketeers, despite the horrors we've seen in places like Texas:

The horrors currently unfolding in Texas expose both the reality of the climate crisis and the extreme vulnerability of fossil fuel infrastructure in the face of that crisis. So of course the Green New Deal finds itself under fierce attack.

Another cousin, the writer Jonathan Neale, has published a new book, Fight the Fire, which describes the Green New Deal in accessible terms. In some ways it's the antithesis of the market-driven approach espoused by businessmen like Bill Gates; it's also a realistic approach, endorsed by climate scientists and academics around the world. You can download it for free from The Ecologist (no registration required). It's worth reading - particularly if you're a skeptic, or looking for a way to share these ideas.

There are still a lot of reasons to hope. The activist Greta Thunberg is one of my heroes: both for her rhetoric and her ability to galvanize an entire generation. In a lot of ways, my young cousin's reaction is also positive; it shows an awareness of the problem, and is certainly more realistic than those who seek to gloss over it.

But we have to act; we have to act now; and we've got to do a lot more than just wait for the market to respond. The invisible hand of the market will see us all killed.


A social network entirely about making, keeping, ...

A social network entirely about making, keeping, and sharing food.

A social network entirely about making, keeping, and sharing food.

Tuesday, 23. February 2021

Simon Willison

Quoting Mike Bostock

When building a tool, it’s easy to forget how much you’ve internalized: how much knowledge and context you’ve assumed. Your tool can feel familiar or even obvious to you while being utterly foreign to everyone else. If your goal is for other people to use the darn thing — meaning you’re not just building for yourself, or tinkering for its own sake (which are totally valid reasons) — you gotta hel

When building a tool, it’s easy to forget how much you’ve internalized: how much knowledge and context you’ve assumed. Your tool can feel familiar or even obvious to you while being utterly foreign to everyone else. If your goal is for other people to use the darn thing — meaning you’re not just building for yourself, or tinkering for its own sake (which are totally valid reasons) — you gotta help people use it! It doesn’t matter what’s possible or what you intended; all that matters is whether people actually succeed in practice.

Mike Bostock


Doc Searls Weblog

Welcome to the 21st Century

Historic milestones don’t always line up with large round numbers on our calendars. For example, I suggest that the 1950s ended with the assassination of JFK in late 1963, and the rise of British Rock, led by the Beatles, in 1964. I also suggest that the 1960s didn’t end until Nixon resigned, and disco took off, […]

Historic milestones don’t always line up with large round numbers on our calendars. For example, I suggest that the 1950s ended with the assassination of JFK in late 1963, and the rise of British Rock, led by the Beatles, in 1964. I also suggest that the 1960s didn’t end until Nixon resigned, and disco took off, in 1974.

It has likewise been suggested that the 20th century actually began with the assassination of Archduke Ferdinand and the start of WWI, in 1914. While that and my other claims might be arguable, you might at least agree that there’s no need for historic shifts to align with two or more zeros on a calendar—and that in most cases they don’t.

So I’m here to suggest that the 21st century began in 2020 with the Covid-19 pandemic and the fall of Donald Trump. (And I mean that literally. Social media platforms were Trump’s man’s stage, and the whole of them dropped him, as if through a trap door, on the occasion of the storming of the U.S. Capitol by his supporters on January 6, 2021. Whether you liked that or not is beside the facticity of it.)

Things are not the same now. For example, over the coming years, we may never hug, shake hands, or comfortably sit next to strangers again.

But I’m bringing this up for another reason: I think the future we wrote about in The Cluetrain Manifesto, in World of Ends, in The Intention Economy, and in other optimistic expressions during the first two decades of the 21st Century may finally be ready to arrive.

At least that’s the feeling I get when I listen to an interview I did with Christian Einfeldt (@einfeldt) at a San Diego tech conference in April, 2004—and that I just discovered recently in the Internet Archive. The interview was for a film to be called “Digital Tipping Point.” Here are its eleven parts, all just a few minutes long:

01 https://archive.org/details/e-dv038_doc_…
02 https://archive.org/details/e-dv039_doc_…
03 https://archive.org/details/e-dv038_doc_…
04 https://archive.org/details/e-dv038_doc_…
05 https://archive.org/details/e-dv038_doc_…
06 https://archive.org/details/e-dv038_doc_…
07 https://archive.org/details/e-dv038_doc_…
08 https://archive.org/details/e-dv038_doc_…
09 https://archive.org/details/e-dv038_doc_…
10 https://archive.org/details/e-dv039_doc_…
11 https://archive.org/details/e-dv039_doc_…

The title is a riff on Malcolm Gladwell‘s book The Tipping Point, which came out in 2000, same year as The Cluetrain Manifesto. The tipping point I sensed four years later was, I now believe, a foreshadow of now, and only suggested by the successes of the open source movement and independent personal publishing in the form of blogs, both of which I was high on at the time.

What followed in the decade after the interview were the rise of social networks, of smart mobile phones and of what we now call Big Tech. While I don’t expect those to end in 2021, I do expect that we will finally see  the rise of personal agency and of constructive social movements, which I felt swelling in 2004.

Of course, I could be wrong about that. But I am sure that we are now experiencing the millennial shift we expected when civilization’s odometer rolled past 2000.


Ally Medina - Blockchain Advocacy

Letter to Attorney General Becerra Re: FinCen Proposed Rule Privacy concerns

February 22, 2021 The Honorable Xavier Becerra California State Capitol SENT VIA EMAIL Dear Attorney General Becerra, On behalf of the Blockchain Advocacy Coalition, an organization of blockchain and virtual currency businesses in California, I write to bring to your attention a pending federal regulation that would preempt and refute many of the important privacy pro

February 22, 2021

The Honorable Xavier Becerra

California State Capitol

SENT VIA EMAIL

Dear Attorney General Becerra,

On behalf of the Blockchain Advocacy Coalition, an organization of blockchain and virtual currency businesses in California, I write to bring to your attention a pending federal regulation that would preempt and refute many of the important privacy protections your office has led the nation on. On December 18, 2020 the US Treasury led by Steve Mnuchin, released a concerning proposed rule that would put into place first of its kind reporting requirements for virtual currencies and digital assets. The agency initially proposed a 15 day comment period over the holidays due to unsubstantiated ‘national security concerns’. After widespread pushback from private citizens, virtual currency companies and members of Congress, the Treasury Department provided another 15 days for reporting requirements and an additional 45 for recordkeeping and counterparting reporting. Fortunately the Biden administration, faced with an avalanche of such poorly thought out rules, gave a 60 day pause and extension on the rulemaking and now the industry is facing a March 1st deadline to comment on a rule that would significantly stifle innovation, limit access to these new products and massively extend the reach of government surveillance of financial transactions far beyond the Bank Secrecy Act (BSA).

If it were to become policy, this rule would preempt California’s consumer privacy laws, significantly weakening the data privacy protections around financial information voters deemed important when approving the Californian Privacy Rights Act in November of 2020. While many parties have opined on the slapdash process and lack of clarity in the proposed rule, we do not believe that the blatant and far reaching consumer privacy implications have been brought to attention . Your office has led the charge implementing and enforcing the nation’s first and strongest consumer privacy framework, particularly for sensitive financial information. Because of this, we wanted to raise the following concerns with the proposed FinCEN rulemaking and ask for your action. The proposed rule complements existing BSA requirements applicable to banks and MSBs (money service business) by proposing to add reporting requirements for virtual currency transactions exceeding $10,000 in value. Pursuant to the proposed rule, banks and MSBs will have 15 days from the date on which a reportable transaction occurs to file a report with FinCEN. Further, this proposed rule would require banks and MSBs to keep records

of a customer’s virtual currency transactions and counterparties, including verifying the identity of their customers, if a counterparty uses an unhosted or otherwise covered wallet and the transaction is greater than $3,000.

Our concerns with the consumer privacy implications of this proposed rule are twofold:

First, the proposed rule’s requirement that MSB’s collect identifying information associated with wallet addresses will create reporting that extends well beyond the intent of the rule or the transaction. According to the EFF “For some cryptocurrencies like Bitcoin, transaction data — including users’ Bitcoin addresses — is permanently recorded on a public blockchain. That means that if you know the name of the user associated with a particular Bitcoin address, you can glean information about all of their Bitcoin transactions that use that address.” California consumers do not have the expectation that a future reporting requirement will link to their entire financial transaction history from that wallet.

Second, this rule creates requirements for disclosure of counterparty information beyond what the BSA requires banks and other financial institutions to collect. It wouldn’t only require these businesses to collect information about their own customers, but also the information of anyone who transacts with those customers using their own cryptocurrency wallets. Specifically:

The name and physical address of each counterparty to the transaction of the financial institution’s customer; Other counterparty information the Secretary may prescribe as mandatory on the reporting form for transactions subject to reporting pursuant to § 1010.316(b); Any other information that uniquely identifies the transaction, the accounts, and, to the extent reasonably available, the parties involved;

Unlike KYC (know your customer) requirements which arise from a direct customer relationship, KYCC (know your customer’s counterparty) requirements unreasonably obligate non-customers to provide personally identifying information to a VASP/MSB (virtual asset service provide/money services business) they do not know or do business with, and whose security and privacy practices they have not evaluated, simply because they happen to transact with one of its customers.

In its haste, the Treasury did not adequately consider the impact of these rules on consumer privacy for those that choose to use virtual currency and would create large scale government surveillance of small personal transactions. We call upon your leadership and expertise in this space to once again lead the charge for consumer protections and submit a comment letter opposing these portions of the proposed rule. Thank you for your consideration and please do not hesitate to reach out with any questions.

Kind Regards,

Ally Medina

Director, Blockchain Advocacy Coalition


FACILELOGIN

The Next TCP/IP Moment in Identity

Loved reading the book Ask Your Developer: How to Harness the Power of Software Developers and Win in the 21st Century by Jeff Lawson. Jeff says in the book that every company is on a journey to becoming a software company and everyone is starting to see the world through the lens of software. He defines the term, software person. A software person is not necessarily a developer, it’s anybod

Loved reading the book Ask Your Developer: How to Harness the Power of Software Developers and Win in the 21st Century by Jeff Lawson.

Jeff says in the book that every company is on a journey to becoming a software company and everyone is starting to see the world through the lens of software. He defines the term, software person. A software person is not necessarily a developer, it’s anybody who, when faced with a problem, asks the question, how can software solve this problem?

Build vs. Buy (or vs. Die)

In the book, Jeff takes the popular debate, build vs. buy, to another dimension; build vs die. As every company is becoming a software company, the competitive advantage they build is in the software they build. When software becomes the interface where the services you offer, meet the customers; unless you build it in the way you want; you die!

Building what you want gives you the freedom to experiment (or innovate). More you experiment or the ability to experiment more, gives you the edge to understand your customers more. Hence, you grow your business.

Build, does not necessarily mean building everything from scratch. You don’t build anything that already exists, given that it provides what you need. You only build things that are core to your business, which help building your competitive advantage over all the others. The rest, or the building blocks that help you build what you wish are part of the digital supply chain.

The Digital Supply Chain

Uber, for example, uses 4000+ microservices internally. However, not all of them are developed by Uber itself. Uber uses Google Maps API to pull out location data, the Twilio API to facilitate communication between passengers and drivers and many other APIs. All these APIs are coming from the digital supply chain Uber picks to build its product. Then again, these building blocks in Uber’s digital supply chain are also available to Lyft, and other Uber competitors around the world. What brings Uber the competitive advantage is in what they build!

The software you build, can be your product, at the same time it can be a building block for another product. Google Maps is Google’s product, however the Google Maps API is a building block for Uber. Alexa is a product of Amazon, however Alexa API is a a building block for Nissan.

Picking the right digital supply chain is equally important as what you pick to build. Think, what if Uber had to build something equivalent to Google Maps from the scratch? From 2016 to 2018, Uber paid 58M USD to Google for using Google Maps. But, then again it’s a peanut, when you compare that with their revenue in 2019, which was 14.15 billion USD.

Having the right digital supply chain helps you to optimize your development team to build only what you need and no more. Instagram, for example, was only a 13 people team, when Facebook acquired it for $1B in 2012; and WhatsApp team was only 50, when Facebook acquired it for $19B in 2014.

Build Your Own Identity Stack?

Every service you develop, every API you design, every device you use, every person you interact with, will have a managed identity, and in today’s hyperconnected world, the Identity integrations with these business applications and systems, is going to be critical.

Going back to the build vs. die debate; do you still have to build the Identity stack to gain the competitive advantage in your business? If you are in the Identity business, of course yes, for all the others no. Identity stack you need to build your product is a building block in the digital supply chain.

You never worried about building a TCP/IP stack yourself, so, don’t worry about building an Identity stack yourself. However, over the time we have spoken to over a thousand companies (hundreds of them are WSO2 customers), and in most of the cases they bring in unique identity requirements. The uniqueness comes in those requirements are specific to the industry they are in and also specific to the complexity of the business problem they want to solve.

Identity is core to any business, and how you manage identity will also help you in building competitive advantage. At WSO2, we have worked with 90% of the Identity Server customers to solve complex identity problems. Identity Server is open source, and if the business problem is straightforward, they don’t even talk to us, they simply use the product as it is. However, when we work with complex Identity requirements, we have extended the product to solve specific business problems.

Building these extensions, specific to unique business requirements helped companies to differentiate themselves from others. Then again, they didn’t want to build everything from scratch — rather they started with what’s common (and available to everyone) and started innovating on that. That drastically reduced the time-to-market, and also gave the freedom to innovate.

I don’t intend to contradict with what I mentioned before, that the Identity stack is part of the digital supply chain you pick, however, the Identity stack you pick for the digital supply chain should have the flexibility to extend with minimal effort to build business requirements specific to your business.

The TCP/IP Moment in Identity

In the 70’s, having support for TCP/IP in a product was considered to be a competitive advantage. Today, it’s given, and nobody worries about TCP/IP support; it’s everywhere.

Ian Glazer from Salesforce, mentioned in his keynote at the European Identity Conference 2016 that, it’s the TCP/IP moment in Identity now. He in fact talked about the open standards (SAML, OpenID Connect, OAuth, SCIM, XACML and so on) in the Identity domain, and how they are going to be part of every product, so no Identity vendor is going to gain competitive advantage just by supporting the open standards. RFPs looking for Identity products will not even worry about asking support for these open standards.

The Next TCP/IP Moment in Identity

Developers do not worry about building a TCP/IP stack, or even worrying about TCP/IP while building software. We believe, the Identity integrations with business applications and systems need to be developer-focused with the right level of abstractions and tools. And, doing that right, would be the next TCP/IP moment in Identity, that will free the developers from worrying about complexities in Identity integrations.

The Developer-focused IAM

The single Identity administrator role has started diminishing, and the role of the developer is becoming more prominent in Identity integrations. These developers need a better abstraction over core identity concepts; and the developer-focused IAM is the way to realize the next TCP/IP moment in Identity.

In the consumer Identity space, when we talk to customers, they bring in their unique requirements. In many cases they look for a product that can be used to build an agile, event-driven consumer Identity (CIAM) platform that can flex to meet frequently changing business requirements.

A developer-focused IAM product builds an abstraction over the core Identity concepts in the form of APIs and SDKs, provides tools for troubleshooting, has the ability to integrate with the organization’s build pipeline, carries the right level of developer experience and has the ability to extend product’s core capabilities to fit into organization’s complex IAM requirements.

As every company is becoming a software company, and starting to build their competitive advantage on the software they build, the developer-focused IAM will free the developers from inherent complexities in doing Identity integrations. That’s the next TCP/IP moment in Identity!

The Next TCP/IP Moment in Identity was originally published in FACILELOGIN on Medium, where people are continuing the conversation by highlighting and responding to this story.


Simon Willison

Quoting Yoz Grahame

I strongly suspect that the single most impactful thing I did during my 5+ years at Linden Lab was shortly before I left: set up a weekly meeting between a couple of leads from Support and Engineering to go over the top 10 support issues. — Yoz Grahame

I strongly suspect that the single most impactful thing I did during my 5+ years at Linden Lab was shortly before I left: set up a weekly meeting between a couple of leads from Support and Engineering to go over the top 10 support issues.

Yoz Grahame


John Philpin : Lifestream

In LinkedIN you can now do rapid self tests on your knowledg

In LinkedIN you can now do rapid self tests on your knowledge of topics. I just did one on Powerpoint. Apparently I am in the top 5% of more than 3 million people that have done the test. I worry for Powerpoint users. I don’t use Powerpoint.

In LinkedIN you can now do rapid self tests on your knowledge of topics. I just did one on Powerpoint. Apparently I am in the top 5% of more than 3 million people that have done the test.

I worry for Powerpoint users.

I don’t use Powerpoint.

Monday, 22. February 2021

Simon Willison

Fuzzy Name Matching in Postgres

Fuzzy Name Matching in Postgres Paul Ramsey describes how to implement fuzzy name matching in PostgreSQL using the fuzzystrmatch extension and its levenshtein() and soundex() functions, plus functional indexes to query against indexed soundex first and then apply slower Levenshtein. The same tricks should also work against SQLite using the datasette-jellyfish plugin.

Fuzzy Name Matching in Postgres

Paul Ramsey describes how to implement fuzzy name matching in PostgreSQL using the fuzzystrmatch extension and its levenshtein() and soundex() functions, plus functional indexes to query against indexed soundex first and then apply slower Levenshtein. The same tricks should also work against SQLite using the datasette-jellyfish plugin.


Quoting Paul Smith

Business rules engines are li’l Conway’s Law devices: a manifestation of the distrust between stakeholders, client and contractor. We require BREs so that separate business units need not talk to each other to solve problems. They are communication and organizational dysfunction made silicon. — Paul Smith

Business rules engines are li’l Conway’s Law devices: a manifestation of the distrust between stakeholders, client and contractor. We require BREs so that separate business units need not talk to each other to solve problems. They are communication and organizational dysfunction made silicon.

Paul Smith


John Philpin : Lifestream

”Without a comment or dissent, the justices issued a one-l

”Without a comment or dissent, the justices issued a one-line order saying they had denied Trump’s request ….” Without dissent Who saw that coming?

”Without a comment or dissent, the justices issued a one-line order saying they had denied Trump’s request ….”

Without dissent

Who saw that coming?


Simon Willison

Blazing fast CI with pytest-split and GitHub Actions

Blazing fast CI with pytest-split and GitHub Actions pytest-split is a neat looking variant on the pattern of splitting up a test suite to run different parts of it in parallel on different machines. It involves maintaining a periodically updated JSON file in the repo recording the average runtime of different tests, to enable them to be more fairly divided among test runners. Includes a recipe

Blazing fast CI with pytest-split and GitHub Actions

pytest-split is a neat looking variant on the pattern of splitting up a test suite to run different parts of it in parallel on different machines. It involves maintaining a periodically updated JSON file in the repo recording the average runtime of different tests, to enable them to be more fairly divided among test runners. Includes a recipe for running as a matrix in GitHub Actions.

Via @djm_


People, processes, priorities

People, processes, priorities Twitter thread from Adrienne Porter Felt outlining her model for thinking about engineering management. I like this trifecta of "people, processes, priorities" a lot.

People, processes, priorities

Twitter thread from Adrienne Porter Felt outlining her model for thinking about engineering management. I like this trifecta of "people, processes, priorities" a lot.

Sunday, 21. February 2021

reb00ted

Liberté, égalité, reparabilité.

Slowly, but steadily, some of tech’s worst practices are being chipped away. Why France’s New Tech ‘Repairability Index’ Is a Big Deal. Wired.

Slowly, but steadily, some of tech’s worst practices are being chipped away. Why France’s New Tech ‘Repairability Index’ Is a Big Deal. Wired.


Doc Searls Weblog

Radio 2.x

On Quora, somebody asks, How can the radio industry stay relevant in the age of streaming music and podcasts? Here’s my answer: It already is, if you consider streaming music and podcasting evolutionary forms of radio. But if you limit the meaning of radio to over-the-air broadcasting, the relevance will be a subordinate one to what’s happening […]

On Quora, somebody asks, How can the radio industry stay relevant in the age of streaming music and podcasts? Here’s my answer:

It already is, if you consider streaming music and podcasting evolutionary forms of radio.

But if you limit the meaning of radio to over-the-air broadcasting, the relevance will be a subordinate one to what’s happening over streaming, cellular and Internet connections, podcasting, satellite radio, digital audio broadcast (DAB) and various forms of Internet-shared video (starting with, but not limited to, YouTube).

The main way over-the-air radio can remain relevant in the long run is by finding ways for live streams to hand off to radio signals, and vice versa. Very little effort is going into this, however, so I expect over-the-air to drift increasingly to the sidelines, as a legacy technology. Toward this inevitable end, it should help to know that AM is mostly gone in Europe (where it is called MW, for MediumWave). This follows in the tracks of LW (longwave) and to some degree SW (shortwave) as well. Stations on those bands persist, and they do have their uses (especially where other forms of radio and Internet connections are absent); but in terms of popularity they are also-rans.

BUT, in the meantime, so long as cars have AM and FM radios in them, the bands remain relevant and popular. But again, it’s a matter of time before nearly all forms of music, talk and other forms of entertainment and sharing move from one-way broadcast to every-way sharing, based on digital technologies. (Latest example: Clubhouse.)


Simon Willison

Cross-database queries in SQLite (and weeknotes)

I released Datasette 0.55 and sqlite-utils 3.6 this week with a common theme across both releases: supporting cross-database joins. Cross-database queries in Datasette SQLite databases are single files on disk. I really love this characteristic - it makes them easy to create, copy and move around. All you need is a disk volume you can create as many SQLite databases as you like. A lesser kno

I released Datasette 0.55 and sqlite-utils 3.6 this week with a common theme across both releases: supporting cross-database joins.

Cross-database queries in Datasette

SQLite databases are single files on disk. I really love this characteristic - it makes them easy to create, copy and move around. All you need is a disk volume you can create as many SQLite databases as you like.

A lesser known feature of SQLite is that you can run queries, including joins, across tables from more than one database. The secret sauce is the ATTACH DATABASE command. Run the following SQL:

ATTACH 'other.db' AS other;

And now you can reference tables in that database as other.tablename. You can then join against them, combine them with UNION and generally treat them as if they were another table in your first connected database.

I've wanted to add support for cross-database queries to Datasette since May 2018. It took me quite a while to settle on a design - SQLite defaults to only allowing ten databases to be attached together, and I needed to figure out how multiple connected databases would fit with the design of the rest of Datasette.

In the end, I decided on the simplest option that would unlock the feature. Run Datasette with the new --crossdb option and the first ten databases passed to Datasette will be ATTACHed to an in-memory database available at the /_memory URL.

The latest.datasette.io demo now exposes two databases using this feature. Here's an illustrative example query that performs a UNION across the sqlite_master metadata table in two databases:

select 'fixtures' as database, * from [fixtures].sqlite_master union select 'extra_database' as database, * from [extra_database].sqlite_master

Try that out here.

Cross-database queries in sqlite-utils

sqlite-utils offers both a Python library and a command-line utility in one package. I've added ATTACH support to both.

The Python library support looks like this:

db = Database("first.db") db.attach("second", "second.db") # Now you can run queries like this: cursor = db.execute(""" select * from table_in_first union all select * from second.table_in_second """) print(cursor.fetchall())

The command-line tool now has a new --attach option which lets you attach a database using an alias. The equivalent query to the above would look like this:

$ sqlite-utils first.db --attach second second.db ' select * from table_in_first union all select * from second.table_in_second'

This defaults to returning results as a JSON array, but you can add --csv or --tsv or other options to get the results back in different output formats.

A cosmetic upgrade to tags on my blog

I noticed that Will Larson's blog shows little numbers next to the tags indicating how many times they have been used. I really liked that, so I've implemented it here as well.

Each entry (and quotation and link) now gets a block in the sidebar that looks like this:

As a long-time fan of faceted search interfaces I really like this upgrade - it helps indicate at a glance the kind of content I have stashed away in my blog's archive.

Releases this week datasette-json-preview: 0.3 - 2021-02-20
Preview of new JSON default format for Datasette sqlite-utils: 3.6 - 2021-02-19
Python CLI utility and library for manipulating SQLite databases datasette: 0.55 - 2021-02-19
An open source multi-tool for exploring and publishing data datasette-graphql: 1.4 - 2021-02-18
Datasette plugin providing an automatic GraphQL API for your SQLite databases higher-lower: 0.1 - 2021-02-16
Functions for finding numbers using higher/lower download-tiles: 0.4.1 - 2021-02-16
Download map tiles and store them in an MBTiles database TIL this week Handling CSV files with wide columns in Python Using io.BufferedReader to peek against a non-peekable stream Loading radio.garden into SQLite using jq

John Philpin : Lifestream

NYT reporting on the British Supreme Court protecting worker

NYT reporting on the British Supreme Court protecting workers rights - the very rights that California voted AGAINST last November. That Britain is ahead of California in this thinking speaks volumes to the shambles that was California.

NYT reporting on the British Supreme Court protecting workers rights - the very rights that California voted AGAINST last November.

That Britain is ahead of California in this thinking speaks volumes to the shambles that was California.

Saturday, 20. February 2021

John Philpin : Lifestream

Justin Timberlake Apologises to Britney Spears and Janet Jac

Justin Timberlake Apologises to Britney Spears and Janet Jackson for His Ignorance Seriously? He is that blind, stupid and deaf? He was never on my list, so can’t scratch him off. So, need to decide what to call my new list that he is at the top of. Suggestions please.

Justin Timberlake Apologises to Britney Spears and Janet Jackson for His Ignorance

Seriously?

He is that blind, stupid and deaf?

He was never on my list, so can’t scratch him off.

So, need to decide what to call my new list that he is at the top of.

Suggestions please.


In a chat today with a tech support person, who at one point

In a chat today with a tech support person, who at one point wrote; ”Had a Mac for a year and used Safari once - to download Chrome.” 😂

In a chat today with a tech support person, who at one point wrote;

”Had a Mac for a year and used Safari once - to download Chrome.”

😂

Friday, 19. February 2021

Simon Willison

Open source projects should run office hours

Back in December I decided to try something new for my Datasette open source project: Datasette Office Hours. The idea is simple: anyone can book a 25 minute conversation with me on a Friday to talk about the project. I’m interested in talking to people who are using Datasette, or who are considering using it, or who just want to have a chat. I’ve now had 35 conversations and it’s been absolutel

Back in December I decided to try something new for my Datasette open source project: Datasette Office Hours. The idea is simple: anyone can book a 25 minute conversation with me on a Friday to talk about the project. I’m interested in talking to people who are using Datasette, or who are considering using it, or who just want to have a chat.

I’ve now had 35 conversations and it’s been absolutely fantastic. I’ve talked to people in Iceland, Burundi, Finland, Singapore, Bulgaria and dozens of other places around the world. I’ve seen my software applied to applications ranging from historic cemetery records to library collections to open city data. It’s been thrilling.

I’d like to encourage more open source project maintainers to consider doing something similar.

Reasons to do this

A challenge of open source is that it's easy to be starved of feedback. People might file bug reports if something breaks, but other than that it can feel like publishing software into a void.

Hearing directly from people who are using your stuff is incredibly motivational. It’s also an amazing source of ideas and feedback on where the project should go next.

In the startup world “talk to your users and potential customers” is advice that becomes a constant drumbeat… because it’s really effective, but it’s also hard to bring up the courage to do!

Talking to users in open source is similarly valuable. And it turns out, especially in these pandemic times, people really do want to talk to you. Office hours is an extremely low-friction way of putting up a sign that says “let’s have a conversation”.

How I do this: 25 minute slots, via Calendly

I’m using Calendly to make 20 minute slots available every Friday between 9am and 5pm Pacific Time (with 12:30-1:30 set aside for lunch), with a ten minute buffer between slots.

In practice, I treat these as 25 minute slots. This gives me five 5 minutes break in between conversations, and also means it’s possible to stretch to 30 minutes if we get to a key topic just before the time slot ends.

I configured Calendly to allow a maximum of five bookings on any Friday. This feels right to me - conversations with five different people can be pretty mentally tiring, and cutting off after five still gives me a good chance to get other work done during the day.

I use Calendly’s Zoom integration, which automatically sends out a calendar invite to both myself and my conversation partner and schedules a Zoom room that’s linked to from the invite. All I have to do is click the link at the appropriate time.

About one in fifteen conversations ends up cancelled. That’s completely fine - I get half an hour of my day back and we can usually reschedule for another week.

How about making some money?

I’ve been having some fascinating conversations on Twitter recently about the challenges of taking an open source project and turning it into a full-time job, earning a salary good enough to avoid the siren call of working for a FAANG company.

People pointed me to a few good examples of open source maintainers who charge for video conference consulting sessions - Graphile’s Benjie and CSS Wizardry's Harry Roberts both let you book paid sessions with them directly.

I really like this as an opportunity for earning money against an open source project, and I think it could complement office hours nicely: 25 minutes on a Friday free on a first-come, first-served basis could then up-sell to a 1.5 hours paid consulting session, which could then lead to larger consulting contracts.

Try this yourself

If you're tempted to try office hours for your own project, getting started is easy. I'm using the Calendly free plan, but their paid plans (which include the ability to attach Stripe or PayPal payments to bookings) are reasonably priced. I've been promoting my sessions via Twitter, the datasette.io website and the Datasette Newsletter.

This is one of those ideas I wish I'd had sooner. It's quickly become a highlight of my week.


Quoting Sarah Drasner

One of the hardest things I’ve had to learn is that humans aren’t pure functions: an input that works one day and gets one result, then again another day and get an entirely different result. — Sarah Drasner

One of the hardest things I’ve had to learn is that humans aren’t pure functions: an input that works one day and gets one result, then again another day and get an entirely different result.

Sarah Drasner

Thursday, 18. February 2021

John Philpin : Lifestream

“Move To Bin?” Now there’s a change for no reason! I’m

“Move To Bin?” Now there’s a change for no reason! I’m talking to you 🍎

“Move To Bin?”

Now there’s a change for no reason!

I’m talking to you 🍎

Wednesday, 17. February 2021

John Philpin : Lifestream

Tickled me - thankyou Stuart  

Tickled me - thankyou Stuart  

Tickled me - thankyou Stuart

 

Tuesday, 16. February 2021

SSI Ambassador

Self-sovereign identity: Legal compliance and the involvement of governments

SSI — Legal compliance and the involvement of governments This article describes how governments of sovereign states might be involved in building identity ecosystems based on self-sovereign principles and how regulatory conformity of such ecosystems can be achieved. When it comes to identity management the involvement of the government can be a tricky topic. It needs to be involved to enable acce
SSI — Legal compliance and the involvement of governments

This article describes how governments of sovereign states might be involved in building identity ecosystems based on self-sovereign principles and how regulatory conformity of such ecosystems can be achieved.

When it comes to identity management the involvement of the government can be a tricky topic. It needs to be involved to enable access to public services, adapt legislature and guarantee equal access for its citizens. However, it should not be able to control or monitor all aspects and activities of its citizens. Self-sovereign identity (SSI) might for some imply, that a citizen is suddenly able to issue his own ID-card, which isn’t the case. Governments are still the primary source of foundational identities.

The government as issuer of foundational identities

While individuals gain more autonomy with SSI the issuance of national IDs is still the responsibility of the public administration. The Pan Canadian Trust Framework (PCTF) differentiates between foundational and contextual identities.

“A foundational identity is an identity that has been established or changed as a result of a foundational event (e.g., birth, person legal name change, immigration, legal residency, naturalized citizenship, death, organization legal name registration, organization legal name change, or bankruptcy).” PCTF [1]

Hence, the government continues to be the issuer of foundational identities and still holds the authority to revoke these credentials when necessary. However, SSI also enables the usage of other identity providers, which are context dependent — leading to a contextual identity as further explained within the PCTF.

“A Contextual Identity is an identity that is used for a specific purpose within a specific identity context (e.g., banking, business permits, health services, drivers licensing, or social media). Depending on the identity context, a contextual identity may be tied to a foundational identity (e.g., a drivers licence) or may not be tied to a foundational identity (e.g., a social media profile).“ [1]

This means a customer of a bank can use his verified bank ID to identify himself at a credit bureau. Since the bank ID is based on a foundational identity, the contextual identity provided by the bank can be sufficient in this particular use-case given the regulatory environment allows such a usage. However, a contextual identity can, but doesn’t have to be based on a foundational identity.

The European Commission supports the continued usage of contextual identities online and only demands the usage of foundational identities when required by law as stated in the eIDAS public consultation [2] regarding the option to extend the regulation for the public sector:

“A European identity solution enabling trusted identification of citizens and companies in their digital interactions to access public or private online services (e.g. e- commerce), should be entirely voluntary for users to adhere to and fully protect data and privacy. Anonymity of the internet should be ensured at all times by allowing solutions for anonymous authentication anonymously where user identification is not required for the provision of the service.“ [2]
Regulatory compliancy

When dealing with personally identifiable information (PII) all involved stakeholders need to adhere to a certain set of laws, which dictate the usage of the such data. These laws highly depend on the citizenship of the data subject (the individual the PII is about) among other important factors. Although the following paragraphs are specifically devoted to the laws of the European Union, the conclusion might also be applied to similar laws such as the California Consumer Privacy Act (CCPA) or the Indian Personal Data Protection Bill (DPA).

Within the European Union, there are two laws, which have a significant influence on identity frameworks. The General Data Protection Regulation, better known as GDPR, determines how personal data from EU citizens can be collected and used. The other important law is the Electronic IDentification, Authentication and trust Services (eIDAS) provision specified in N°910/2014 [3]. It constitutes the main electronic identification trust framework in the EU and is an elemental building block of the digital single market.

GDPR — General Data Protection Regulation Overview of important roles regarding PII data management according to international standards.

The EBSI GDPR assessment [4] notes that

“According to this Regulation, there are two types of actors whose key role in data processing and whose relationship to the data within the data processing environment leads the European legislator to attribute them a set of obligations and responsibilities. Thus, these liable actors are subject to data protection rules.“ [4]

These are data controllers, which are defined in article 4(7) GDPR [5] as

“the natural or legal person, public authority, agency or other body which, alone or jointly with others, determines the purposes and means of the processing of personal data; where the purposes and means of such processing are determined by Union or Member State law, the controller or the specific criteria for its nomination may be provided for by Union or Member State law” [5]

which

“have to take all necessary measures so that data subjects are sufficiently informed and have the ability to exercise their data protection rights.“ [4]

The other actor is the data processor who acts as delegate of the data controller and is a separate legal entity according to the opinion 1/2010 of the Data protection working party. [6] With multiple nodes running a decentralized network every node acts as a data processor or data controller depending on if the node operator is processing the data as a delegate or not. The EBSI GDPR report further notes:

“in case of joint controllership, data controllers can contractually assign partial responsibility based on distinct stages of data processing.“ While an agreement between these data processors can regulate the responsibilities, “data subjects will have ot (sic!) be able to exercise their rights against every joint controller“ and “nodes that add and process the on-chain ledger data in order to maintain the consensus will be individually qualified as joint data controllers and this, regardless of a contractual relationship stating the contrary.“ [4]

For public blockchains with permissionless write access such as Bitcoin or Ethereum, this means, that every miner, which is participating in the proof of work consensus is regarded as data processor given there is an unintentional personal data leakage or correlation with an URL (Uniform resource locator) of a service endpoint within DID Documents as pointed out as critical to keep PII private by the DID specification of the W3C in section 10.1.[7] This threat in addition of the numerous other correlation risks mentioned in section 10.2 and 10.3 of said specification make the current implementation of SSI based on permissionless blockchains, which inhibit the capability for natural persons to write a anywise DID on the ledger, a daunting privacy challenge.

Another important aspect is the question if credentials (or any other form of PII) is stored as hash on the verifiable data registry. A hash is a data digest and is considered a one-way function, which in theory leads to an anonymization of the original information. The debate around the question if a hash constitutes PII is likely to continue, since national data protection agencies are struggling to clearly define if the hashing can be considered an anonymization or pseudonymization.

“According to the Spanish DPA (Data protection agency), hashing can at times be considered as anonymization or pseudonymization depending on a variety of factors varying from the entities involved to the type of the data at hand.“ [4]

Even if the hash constitutes a one-way obfuscation technique, which anonymizes PII, it I) requires a transaction on a public ledger and II) it puts data controllers in a higher risk position with the obligation to avoid correlation of individuals. Risk minimizing obligations for data controllers are easier to implement when there is no hash of a verified credential or verified presentation stored on a public ledger.

When it comes to the wallet itself the EBSI GDPR report notes that

“there is growing consensus about the possibility of data subjects to being simultaneously considered as data controllers for the data that refer to themselves“. The report provides the recommendation that “the privacy preserving technical and organisational measures of the wallet and the personal data transmissions should ensure that the necessary safeguards are in place in order to not limit the empowerment of the data subject through the DLT chosen model.“ [4]

The report concludes, that data within the wallet application is considered personal data and therefor is subject to the data protection regulation.

While there is a general assumption that e.g. Hyperledger Indy implementations are GDPR compliant [8], ultimately courts have to decide if that claim holds up based on a case by case evaluation on the particular implementation. Nevertheless, avoiding the exposure of PII on the verifiable data registry, by I) not allowing natural persons to write public DIDs and II) not storing PII in hashed form on the verifiable data registry facilitate the GDPR compliance obligations.

eIDAS:

The eIDAS regulation [3] is concerned with two distinct topics. One part is concerned with trust services for private businesses such as electronic signatures, seal, time stamps etc. The other part is regulating the mutual recognition among member states of national implementations of electronic identification (eID) for the public sector. Is a technology neutral approach, which has a strong influence on the international regulatory space. The main goal of mutual recognition of eID is to enable EU citizens access to cross-border public services with their own national eID means. The implementation of eID schemes vary from member state to member state and not all member states have notified an eID scheme as illustrated by the overview of pre-notified and notified eID schemes [9] under eIDAS.

There are three levels of assurance specified for eIDs under eIDAS referring to the degree of confidence in the claimed identity of a person, which include detailed criteria allowing member states to map their eID means against a benchmark (low, substantial and high). Current SSI implementations have the objective to be recognized with a level of assurance specified as substantial.

It’s currently possible to be eIDAS compliant with SSI by leveraging one out of five scenarios described in the SSI eIDAS legal report by Dr. Ignacio Alamillo Domingo [10]. Especially interesting is the SSI eIDAS bridge, which adds legal value to verified credentials with the use of electronic certificates and electronic seals. However, it’s also possible to derive national eIDs notified in eIDAS, which are eIDAS linked by deriving a national eID by issuing a verifiable credential with a qualified certificate according to the technical specification.[12]

Nevertheless, there are also hindrances in the process of creating a qualified certificate with the derived national identity, because of the way the regulation is defining a qualified signature. Another issue is that national eID schemes require the keys to be in a secure element. However, current SSI wallets only offer software keys and do not leverage the security benefits of a hardware element. Furthermore, the eIDAS regulation doesn’t regulate the case of a private entity issuing an eID attribute to a natural person for the usage of it in other private interactions.

Furthermore, the authentication process to achieve the recognition of notified eIDAS schemes by other member states requires a national node, which provides the authentication service. While aimed to be technology neutral, the obligation to provide this authentication service as delegated authentication component has several drawbacks and also hinders the potential adoption of SSI. The EU has already identified the need to re-evaluate the policies set by eIDAS.

“Fundamental changes in the overall societal context suggest a revision of the eIDAS Regulation. These include a dramatic increase in the use of novel technologies, such as distributed-ledger based solutions, the Internet of Thing, Artificial Intelligence and biometrics, changes in the market structure where few players with significant market power increasingly act as digital identity ‘gatekeepers’, changes in user behavior with increasing demand for instant, convenient and secure identification and the evolution of EU Data Protection legislation” [2]

The consultation continues with its target:

“The objective of this initiative is, first of all, to provide a future proof regulatory framework to support an EU-wide, simple, trusted and secure system to manage identities in the digital space, covering identification, authentication and the provision of attributes, credentials and attestations. Secondly, the initiative aims at creating a universal pan-European single digital ID. These objectives could be achieved through an overhaul of the eIDAS system, an extension of eIDAS to the private sector, the introduction of a European Digital Identity (EUid) building on the eIDAS system or combination of both.” [2]

In an private interview for an academic study Dr. Ignacio Alamillo Domingo suggested embodying new technologies such as SSI into the revised regulation e.g. by not mandating the provision of an authentication facility and creating new trust services such as electronic identification. In another private interview Luca Boldrin suggested keeping national identity systems as they are but use a derivation of national identity for cross-border context for public and private businesses in parallel to current node implementation to enable a European identity.

Dr. Ignacio Alamillo Domingo argues that having derived national eIDs and eID trust services has the benefit of increased privacy by using a peer to peer authentication instead of a delegated authentication model (the national eIDAS node). This also leads to less liability issues by shifting the authentication part to private providers as well as less costs associated with running authentication infrastructure for governments, because these are provided by Distributed Public Key Infrastructure (DPKI) instead of national eIDAS nodes. These DPKI systems also have the benefits of being more resilient to attacks compared to a single node, which represents a single point of failure. However, regulating eID as trust service also means opening up identification for the private market, which might not be in the interest of national governments.

Disclaimer: This article does not represent the official view of any entity, which is mentioned in this article or which is affiliated with the author. It solely represents the opinion of the author.

SSI Ambassador
Adrian Doerk
Own your keys

Sources:

[1] PSP PCTF Working Group. ‘Pan Canadian Trust Framework (PCTF) V1.1’. GitHub, 2. June 2020. Accessed 22. June 2020. https://github.com/canada-ca/PCTF- CCP/blob/master/Version1_1/PSP-PCTF-V1.1-Consultation-Draft.pdf

[2] European Commission, ‘EU Digital ID Scheme for Online Transactions across Europe, Public Consultation, Inception Impact Assessment — Ares(2020)3899583’ Accessed 23. August 2020

[3] European parliament and the council of the European union. REGULATION (EU) No 910/2014 on electronic identification and trust services for electronic transactions in the internal market and repealing Directive 1999/93/EC (2014). Accessed 20. September 2020. https://ec.europa.eu/futurium/en/system/files/ged/eidas_regulation.pdf

[4] CEF Digital, University of Amsterdam. ‘EBSI GDPR Assessment, Report on Data Protection within the EBSI Version 1.0 Infrastructure.’ CEF Digital, April 2020. Accessed 18. August 2020. https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITALEBSI/Legal+Assessment+Reports

[5] ‘REGULATION (EU) 2016/679 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL (GDPR)’, 2016. Accessed 5. September 2020. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0679

[6] Working Party up under Article 29 of Directive 95/46/EC. ‘Article 29 Data Protection Working Party, “Opinion 1/2010 on the Concepts of ‘Controller’ and ‘Processor’” (2010)’, 16 February 2010. Accessed 5. September 2020. https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2010/wp169_en.pdf

[7] World Wide Web Consortium (W3C), ‘Decentralized Identifiers (DIDs) v1.0’. Accessed 18. August 2020. https://www.w3.org/TR/did-core/

[8] Sovrin Foundation. ‘GDPR Position Paper: Innovation Meets Compliance’, January 2020. Accessed 5. September 2020. https://sovrin.org/wp-content/uploads/GDPR-Paper_V1.pdf

[9] CEF Digital ‘Overview of Pre-Notified and Notified EID Schemes under EIDAS’, 2. January 2019. Accessed 6. September 2020. https://ec.europa.eu/cefdigital/wiki/display/EIDCOMMUNITY/Overview+of+pre- notified+and+notified+eID+schemes+under+eIDAS

[10] Dr. Ignacio Alamillo Domingo. ‘SSI EIDAS Legal Report’, April 2020, 150. Accessed 14. September 2020. https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITALEBSI/Legal+Assessment+Reports

[11] EBSI, ESSIF. ‘Technical Specification (15) — EIDAS Bridge for VC-ESealing’. CEF Digital, n.d. Accessed 2. September 2020. https://ec.europa.eu/cefdigital/wiki/cefdigital/wiki/display/CEFDIGITALEBSI/Technical+Specification+%2815%29+-+eIDAS+bridge+for+VC-eSealing

Monday, 15. February 2021

Jon Udell

My print edition superpower

When I walk around our neighborhood I see very few copies of our local newspaper on sidewalks, porches, and front lawns. We subscribe because it’s a lifelong habit, and because we like to support local journalism, and in my case because it’s a welcome reprieve from a long day of screentime. This archaic habit has … Continue reading My print edition superpower

When I walk around our neighborhood I see very few copies of our local newspaper on sidewalks, porches, and front lawns. We subscribe because it’s a lifelong habit, and because we like to support local journalism, and in my case because it’s a welcome reprieve from a long day of screentime.

This archaic habit has also, weirdly, become a kind of superpower. From time to time I meet people who are surprised when I know facts about them. These facts are public information, so why should it be surprising that I know them? Because they appear in the print edition of the newspaper. Consider the man in this photo.

We were out for a Sunday morning walk, carrying the newspaper, when we saw him in a cafe. We had just read an article about Whiskerino, the annual contest of fancy beards and mustaches in Petaluma. And here was the guy whose photo was featured in the article!

That was a nice coincidence, but the kicker is that he had no idea his picture was in the paper. Not being of newspaper-reading age, and not having friends of newspaper-reading age, he had to depend on elders to alert him. I guess we got to him before his parents did.

When a similar thing happened this weekend, it occurred to me that this could be a good marketing strategy for newspapers. Do you want to wield an information superpower? Do you want to amaze people by knowing facts that they can’t imagine you could possibly know? Subscribe to the print edition of your local newspaper!

Sunday, 14. February 2021

Identity Woman

My Articles On DIF

In 2020 I had a contract along with Juan Caballero to do communications at DIF for a few months. We got the youtube channel going with content from the F2F event and published several articles. I coauthored this one with Margo Johnson about the glossary process we went through to define Wallet, Agent and Credential. […] The post My Articles On DIF appeared first on Identity Woman.

In 2020 I had a contract along with Juan Caballero to do communications at DIF for a few months. We got the youtube channel going with content from the F2F event and published several articles. I coauthored this one with Margo Johnson about the glossary process we went through to define Wallet, Agent and Credential. […]

The post My Articles On DIF appeared first on Identity Woman.


Simon Willison

Weeknotes: Finally, an intro video for Datasette

My big project this week was this Video introduction to Datasette and sqlite-utils. I recorded the video a few weeks ago in advance of FOSDEM, but this week I put together the annotated version. I'm really happy with it, and I've added it to the datasette.io homepage as a starting point for helping people understand the project. Annotating the video I'm not a huge watcher of video tutorials -

My big project this week was this Video introduction to Datasette and sqlite-utils. I recorded the video a few weeks ago in advance of FOSDEM, but this week I put together the annotated version. I'm really happy with it, and I've added it to the datasette.io homepage as a starting point for helping people understand the project.

Annotating the video

I'm not a huge watcher of video tutorials - I'm impatient and find myself watching them at double speed and still complaining that a text version would be more efficient. So when I publish my own videos I like to accompany them with a useful text version.

The format I've settled on - for this video and for others like Personal Data Warehouses: Reclaiming Your Data - is to use screenshots from the video accompanied by notes, links and code samples. Ideally you can read the text version without watching the video at all, but if you do watch the video the text version can provide extended annotations.

I created this one on macOS using the combination of QuickTime Player and Preview. If you hit Command+C while watching a video in QuickTime Player a PNG snapshot gets copied to your clipboard. Switch to Preview with Command+Tab and hit Command+N to create a new untitled document there containing the image.

I ran through the video creating snapshots of each interesting moment in this way, leaving 40+ open Preview documents called "Untitled", "Untitled 2" and so on.

When I reached the end of the video I switched back to Preview and used Command+S to save each open document in turn, creating a folder full of images with names like "Untitled 2.png". These were in the correct order.

I used the Finder "Rename 40 items..." right-click menu option to remove the Untitled prefix.

Then I optimized the PNGs using the tricks described in this TIL.

Next step: create the HTML. I have a template that I use for each "slide" which looks like this:

<div class="slide"> <img alt="" height="{height}" src="{filename}" width="{width}"> <div> <p>Words</p> </div> </div>

I constructed an Observable notebook which accepts a list of filenames (copied and pasted directly from the Finder) and turns them into a sequence of those HTML templates.

Having built out the HTML framework for the page, the last step was to go through and add the annotations. I did that by editing the HTML directly.

datasette-tiles: OSM v.s. TMS

Matthew Somerville pointed out that my datasette-tiles project (described here previously) had the Y axis flipped from the standard used by Google Maps and OpenStreetMap.

It turns out there are two rival standards for this. TMS - Tile Map Service - is a tile specification developed by the Open Source Geospatial Foundation. It's used by the MBTiles specification which is why datasette-tiles was using it.

Google Maps and OpenStreetMap do things slightly differently - counting Y from the top of the map instead of the bottom. This has become the de facto standard for web mapping. Tom MacWright has published a useful explainer of the diffference between the two.

I made myself a couple of diagrams to ensure I completely understood how the two tile systems work:

datasette-tiles 0.6 ships a breaking change that switches the default serving mechanism to the OpenStreetMap system - so /-/tiles/db/z/x/y.png now serves tiles using that coordinate system.

If you want the TMS mechanism, you can use the new /-/tiles-tms/db/z/x/y.png endpoint instead.

Releases this week evernote-to-sqlite: 0.3.1 - 2021-02-11
Tools for converting Evernote content to SQLite datasette-tiles: 0.6.1 - 2021-02-11
Mapping tile server for Datasette, serving tiles from MBTiles packages TIL this week Shrinking PNG files with pngquant and oxipng

Saturday, 13. February 2021

John Philpin : Lifestream

Mark Zuckerberg Asks His Staff to ‘Inflict Pain’ on Apple Ov

Mark Zuckerberg Asks His Staff to ‘Inflict Pain’ on Apple Over iOS 14 Privacy Dispute I look forward to seeing what that might be. EEJIT

Clubhouse is clearly no Vegas - and this isn’t even the art

Clubhouse is clearly no Vegas - and this isn’t even the article about their infrastructure, the Chinese, personal creds passing through the interwebs in open text. What could possibly go wrong?

Clubhouse is clearly no Vegas - and this isn’t even the article about their infrastructure, the Chinese, personal creds passing through the interwebs in open text.

What could possibly go wrong?


Information Answers

(P)algorithms

I mentioned the concept of Personal Algorithms, or (P)algorithms back in this post at the start of the Covid pandemic. I think they make for an […]
I mentioned the concept of Personal Algorithms, or (P)algorithms back in this post at the start of the Covid pandemic. I think they make for an […]

Friday, 12. February 2021

John Philpin : Lifestream

The Currency of Life has moved from engagement - to attentio

The Currency of Life has moved from engagement - to attention - to intention. Do we agree?

The Currency of Life has moved from engagement - to attention - to intention.

Do we agree?


So Much Truth In This

So Much Truth In This

So Much Truth In This


MyDigitalFootprint

The New Fatigue - what is this all about?

Not sure about you but there is something different about the current situation (Feb 2021).  I wrote about the  7B’s as our responses to lockdown and I believe still that viewpoint stands. However, there is something new, right now, which is different it is a mistiness,  a malaise, fatigue, sapped, brain fog.  To be clear, this is not a loss of motivation, depression or oth


Not sure about you but there is something different about the current situation (Feb 2021).  I wrote about the  7B’s as our responses to lockdown and I believe still that viewpoint stands. However, there is something new, right now, which is different it is a mistiness,  a malaise, fatigue, sapped, brain fog.  To be clear, this is not a loss of motivation, depression or other mental issues - which are all very real and getting to everyone but I am talking about something else.

In 1996 (web 1.0) the market discovered the opportunity for an online market, everyone took their physical business and replicated it exactly (give or take a bit) to work in a web browser.  On-line arrived.  We quickly worked out that this was an unmitigated disaster as a user experience, operationally and back office was a mess.  Come 2001 post-crash we had stopped taking off-line thinking and plonking it on-line and saying to ourselves, this is fab.  We started digital-first.  

2020 we were forced to take the remains of all offline ways of working and stuff it online - meetings, innovation, communication, management, reporting, selling, socialising, dating, laughing, onboarding, education, training - everything else we did in person and face to face.

We have just replicated the 1996 sorry mess - we imaged that we could take what was offline and stuff it into the tech available and it would be magic. My fatigue I am sure is because it has not been designed digital-first, to be specific I mean the workflow, UI and UX. Starting from “what work is needed” and not starting from “does this great scalable platform do the job!”

Perhaps we need to stop, look in the mirror and say to ourselves: “look this pandemic is not going away.”  Then we need to work out what work needs to be done and how we can do it digital-first rather than pretending it is all ok and that we can ignore it as we will all go back to normal - please no.  Just expecting everyone to buckle down and get on is no longer a good strategy. 

I ask three questions

Is this working for us? - No it is not working, the malaise, fatigue and brain fog are a clue.  We got away with it for a time, but this is not a long term solution

Who is us?  - those working from home but the work is more than working at home, we have lost our connectedness, bonding and togetherness

Of whom are we asking the question? - Senior leadership and Directors who owe a duty of care and responsibility. It is now obvious that we have to reimagine work starting from digital-first for all those aspects of our working lives that we had left as they were too hard and difficult.   






Ludo Sketches

ForgeRock Directory Services 7

In August 2020, we’ve rolled out a new release of the ForgeRock Identity Platform which included updated versions of all of the products, including Directory Services 7.0. I didn’t write a post about the new release, mostly due to our… Continue reading →

In August 2020, we’ve rolled out a new release of the ForgeRock Identity Platform which included updated versions of all of the products, including Directory Services 7.0. I didn’t write a post about the new release, mostly due to our focus to deliver the ForgeRock Identity Cloud and family vacation.

But ForgeRock Directory Services 7.0 is a major release in many ways. It is the first to be released with a sample docker file and full support to run in Kubernetes environments in the Cloud. To achieve that, we’ve made a number of significant changes especially in how security is managed, and how replication is configured and enabled. The rest of the server remains quite the same, delivering consistent performance and reliability. You should read the release notes for all the details.

Since, DS 7 was successfully deployed in production, in VMs or in Docker/Kubernetes, and our customers have praised the simplicity and efficiency of the new version. However, some customers have experienced some difficulties with upgrading their current deployment to the 7.0 release, mostly due to the changes I’ve mentioned above. So we have been improving our Upgrade Guide, with greater details, and my colleague Mark Craig, has posted a series of 3 articles on Upgrading to Directory Services 7:

What has changed? Upgrading by adding new servers Doing In-place Upgrade

If you’re planning to upgrade an OpenDJ or a ForgeRock Directory Services to the latest release, I would high recommend to read the Directory Services Upgrade Guide, and then Mark’s posts.


Phil Windley's Technometria

Passwords Are Ruining the Web

Summary: Passwords are ruining the web with awful, lengthy, and inconsistent user experiences. They're insecure and lead to data breaches. The good news is there are good ways for web sites to be passwordless. If you hate passwords, build the world you want to live in. Compare, for a moment, your online, web experience at your bank with the mobile experience from the same bank. Chances

Summary: Passwords are ruining the web with awful, lengthy, and inconsistent user experiences. They're insecure and lead to data breaches. The good news is there are good ways for web sites to be passwordless. If you hate passwords, build the world you want to live in.

Compare, for a moment, your online, web experience at your bank with the mobile experience from the same bank. Chances are, if you're like me, that you pick up your phone and use a biometric authentication method (e.g. FaceId) to open it. Then you select the app and the biometrics play again to make sure it's you, and you're in.

On the web, in contrast, you likely end up at a landing page where you have to search for the login button which is hidden in a menu or at the top of the page. Once you do, it probably asks you for your identifier (username). You open up your password manager (a few clicks) and fill the username and only then does it show you the password field1. You click a few more times to fill in the password. Then, if you use multi-factor authentication (and you should), you get to open up your phone, find the 2FA app, get the code, and type it in. To add insult to injury, the ceremony will be just different enough at every site you visit that you really don't develop much muscle memory for it.

As a consequence, when I need somethings from my bank, I pull out my phone and use the mobile app. And it's not just banking. This experience is replicated on any web site that requires authentication. Passwords and the authentication experience are ruining the web.

I wouldn't be surprised to find businesses abandon functional web sites in the future. There will still be some marketing there (what we used to derisively call "brochure-ware") and a pointer to the mobile app. Businesses love mobile apps not only because they can deliver a better user experience (UX) but because they allow business to better engage people. Notifications, for example, get people to look at the app, giving the business opportunities to increase revenue. And some things, like airline boarding passes, just work much better on mobile.

Another factor is that we consider phones to be "personal devices". They aren't designed to be multi-user. Laptops and other devices, on the other hand, can be multi-user, even if in practice they usually are not. Consequently, browsers on laptops get treated as less insecure and session invalidation periods are much shorter, requiring people to login more frequently than in mobile apps.

Fortunately, web sites can be passwordless, relieving some of the pain. Technologies like FIDO2, WebAuthn, and SSI allow for passwordless user experiences on the web as well as mobile. The kicker is that this isn't a trade off with security. Passwordless options can be more secure, and even more interoperable, with a better UX than passwords. Everybody wins.

Notes This is known as "identifier-first authentication". By asking for the identifier, the authentication service can determine how to authenticate you. So, if you're using a token authentication instead of passwords, it can present that next. Some places do this well, merely hiding the password field using Javascript and CSS, so that password managers can still fill the password even though it's not visible. Others don't.

Photo Credit: Login Window from AchinVerma (Pixabay)

Tags: identity passwords authentication ssi


John Philpin : Lifestream

and so the decline to oblivion continues ”GOP senators

and so the decline to oblivion continues ”GOP senators praise impeachment managers but say Trump will be acquitted.” The Hill

and so the decline to oblivion continues

”GOP senators praise impeachment managers but say Trump will be acquitted.”

The Hill


Identity Woman

Podcast: Mint & Burn

I had a great time with the the folks at RMIT on their Mint & Burn Podcast. Enjoy! The post Podcast: Mint & Burn appeared first on Identity Woman.

I had a great time with the the folks at RMIT on their Mint & Burn Podcast. Enjoy!

The post Podcast: Mint & Burn appeared first on Identity Woman.


The Flavors of Verifiable Credentials

I have authored a new paper in my new role as Ecosystems Director at CCI. You can read the blog post about it on the Linux Foundation Public Health and download the paper in PDF form here. The post The Flavors of Verifiable Credentials appeared first on Identity Woman.

I have authored a new paper in my new role as Ecosystems Director at CCI. You can read the blog post about it on the Linux Foundation Public Health and download the paper in PDF form here.

The post The Flavors of Verifiable Credentials appeared first on Identity Woman.


Two Exciting New Roles

I should have written this post at the beginning of the year…but the year is still young. I have two new part time roles that I’m really excited about. I am the Ecosystems Director at the Covid-19 Credentials Initiative. I am working with a fantastic team helping lead/organize this community. Lucy Yang is the Community […] The post Two Exciting New Roles appeared first on Identity Woman.

I should have written this post at the beginning of the year…but the year is still young. I have two new part time roles that I’m really excited about. I am the Ecosystems Director at the Covid-19 Credentials Initiative. I am working with a fantastic team helping lead/organize this community. Lucy Yang is the Community […]

The post Two Exciting New Roles appeared first on Identity Woman.

Thursday, 11. February 2021

Simon Willison

Quoting Ben Johnson

Litestream runs continuously on a test server with generated load and streams backups to S3. It uses physical replication so it'll actually restore the data from S3 periodically and compare the checksum byte-for-byte with the current database. — Ben Johnson

Litestream runs continuously on a test server with generated load and streams backups to S3. It uses physical replication so it'll actually restore the data from S3 periodically and compare the checksum byte-for-byte with the current database.

Ben Johnson


trustme

trustme This looks incredibly useful. Run "python -m trustme" and it will create three files for you: server.pem, server.key and a client.pem client certificate, providing a certificate for "localhost" (or another host you spefict) using a fake certificate authority. Looks like it should be the easiest way to test TLS locally. Via Seth Michael Larson

trustme

This looks incredibly useful. Run "python -m trustme" and it will create three files for you: server.pem, server.key and a client.pem client certificate, providing a certificate for "localhost" (or another host you spefict) using a fake certificate authority. Looks like it should be the easiest way to test TLS locally.

Via Seth Michael Larson


Why I Built Litestream

Why I Built Litestream Litestream is a really exciting new piece of technology by Ben Johnson, who previously built BoltDB, the key-value store written in Go that is used by etcd. It adds replication to SQLite by running a process that converts the SQLite WAL log into a stream that can be saved to another folder or pushed to S3. The S3 option is particularly exciting - Ben estimates that keeping

Why I Built Litestream

Litestream is a really exciting new piece of technology by Ben Johnson, who previously built BoltDB, the key-value store written in Go that is used by etcd. It adds replication to SQLite by running a process that converts the SQLite WAL log into a stream that can be saved to another folder or pushed to S3. The S3 option is particularly exciting - Ben estimates that keeping a full point-in-time recovery log of a high write SQLite database should cost in the order of a few dollars a month. I think this could greatly expand the set of use-cases for which SQLite is sensible choice.

Wednesday, 10. February 2021

Simon Willison

Dependency Confusion: How I Hacked Into Apple, Microsoft and Dozens of Other Companies

Dependency Confusion: How I Hacked Into Apple, Microsoft and Dozens of Other Companies Alex Birsan describes a new category of security vulnerability he discovered in the npm, pip and gem packaging ecosystems: if a company uses a private repository with internal package names, uploading a package with the same name to the public repository can often result in an attacker being able to execute th

Dependency Confusion: How I Hacked Into Apple, Microsoft and Dozens of Other Companies

Alex Birsan describes a new category of security vulnerability he discovered in the npm, pip and gem packaging ecosystems: if a company uses a private repository with internal package names, uploading a package with the same name to the public repository can often result in an attacker being able to execute their own code inside the networks of their target. Alex scored over $130,000 in bug bounties from this one, from a number of name-brand companies. Of particular note for Python developers: the --extra-index-url argument to pip will consult both public and private registries and install the package with the highest version number!

Via @jacobian

Tuesday, 09. February 2021

John Philpin : Lifestream

Anyone else think that the ‘’remote work conversation’ has d

Anyone else think that the ‘’remote work conversation’ has disappeared up our own posteriors? All this discussion about remote work and asking if we will return to the old ways or adopt new ones. Has anyone asked people that work in transport, retail, sanitation, manufacturing, construction, schools, hospitals, hospitality. … you know those jobs where there isn’t the choice … what they thin

Anyone else think that the ‘’remote work conversation’ has disappeared up our own posteriors?

All this discussion about remote work and asking if we will return to the old ways or adopt new ones.

Has anyone asked people that work in transport, retail, sanitation, manufacturing, construction, schools, hospitals, hospitality. … you know those jobs where there isn’t the choice … what they think?

The dominant conversation does seem to be about the minority of people.

Again!

Monday, 08. February 2021

Simon Willison

Quoting Jacob Kaplan-Moss

Finally, remember that whatever choice is made, you’re going to need to get behind it! You should be able to make a compelling positive case for any of the options you present. If there’s an option you can’t support, don’t present it. — Jacob Kaplan-Moss

Finally, remember that whatever choice is made, you’re going to need to get behind it! You should be able to make a compelling positive case for any of the options you present. If there’s an option you can’t support, don’t present it.

Jacob Kaplan-Moss


Matt Flynn: InfoSec | IAM

Comprehensive Identity-as-a-Service (IDaaS): Protect all your apps with cloud access management

Comprehensive Identity-as-a-Service (IDaaS): Protect all your apps with cloud access management Over a decade ago, the need for quicker SaaS onboarding led to Siloed IAM for early IDaaS adopters. For many, IDaaS evolved to a Hybrid IAM approach. Today, Oracle’s IDaaS provides comprehensive coverage for enterprise apps.  "IDaaS has matured quite a bit over the last several years and no l

Comprehensive Identity-as-a-Service (IDaaS): Protect all your apps with cloud access management

Over a decade ago, the need for quicker SaaS onboarding led to Siloed IAM for early IDaaS adopters. For many, IDaaS evolved to a Hybrid IAM approach. Today, Oracle’s IDaaS provides comprehensive coverage for enterprise apps. 

"IDaaS has matured quite a bit over the last several years and no longer relies as much on SAML or pre-built app templates. Today, Oracle Identity Cloud Service helps manage access to virtually any enterprise target. To accomplish that, we’ve introduced several technical approaches to bringing more applications into the IDaaS fold with less effort. These approaches, combined, provide the easiest path toward enabling the service to manage access for more systems and applications."

Read more on the Oracle Cloud Security Blog > Comprehensive Identity-as-a-Service (IDaaS): Protect all your apps with cloud access management.


Phil Windley's Technometria

Persistence, Programming, and Picos

Summary: Picos show that image-based development can be done in a manner consistent with the best practices we use today without losing the important benefits it brings. Jon Udell introduced me to a fascinating talk by the always interesting r0ml. In it, r0ml argues that Postgres as a programming environment feels like a Smalltalk image (at least that's the part that's germane to th

Summary: Picos show that image-based development can be done in a manner consistent with the best practices we use today without losing the important benefits it brings.

Jon Udell introduced me to a fascinating talk by the always interesting r0ml. In it, r0ml argues that Postgres as a programming environment feels like a Smalltalk image (at least that's the part that's germane to this post). Jon has been working this way in Postgres for a while. He says:

For over a year, I’ve been using Postgres as a development framework. In addition to the core Postgres server that stores all the Hypothesis user, group, and annotation data, there’s now also a separate Postgres server that provides an interpretive layer on top of the raw data. It synthesizes and caches product- and business-relevant views, using a combination of PL/pgSQL and PL/Python. Data and business logic share a common environment. Although I didn’t make the connection until I watched r0ml’s talk, this setup hearkens back to the 1980s when Smalltalk (and Lisp, and APL) were programming environments with built-in persistence. From The Image of Postgres
Referenced 2021-02-05T16:44:56-0700

Here's the point in r0ml's talk where he describes this idea:

As I listened to the talk, I was a little bit nostalgic for my time using Lisp and Smalltalk back in the day, but I was also excited because I realized that the model Jon and r0ml were talking about is very much alive in how one goes about building a pico system.

Picos and Persistence

Picos are persistent compute objects. Persistence is a core feature of how picos work. Picos exhibit persistence in three ways. Picos have1:

Persistent identity—Picos exist, with a single identity, continuously from the moment of their creation until they are destroyed. Persistent state—Picos have state that programs running in the pico can see and alter. Persistent availability—Picos are always on and ready to process queries and events.

Together, these properties give pico programming a different feel than what many developers are used to. I often tell my students that programmers write static documents (programs, configuration files, SQL queries, etc.) that create dynamic structures—the processes that those static artifacts create when they're run. Part of being a good programmer is being able to envision those dynamic structures as you program. They come alive in your head as you imagine the program running.

With picos, you don't have to imagine the structure. You can see it. Figure 1 shows the current state of the picos in a test I created for a collection of temperature sensors.

Figure 1: Network of Picos for Temperatures Sensors (click to enlarge)

In this diagram, the black lines show the parent-child hierarchy and the dotted pink lines show the peer-to-peer connections between picos (called "subscriptions" in current pico parlance). Parent-child hierarchies are primarily used to manage the picos themselves whereas the heterarchical connections between picos is used for programmatic communication and represent the relationships between picos. As new picos are created or existing picos are deleted, the diagram changes to show the dynamic computing structure that exists at any given time.

Clicking on one of the boxes representing a pico opens up a developer interface that enables interaction with the pico according to the rulesets that have been installed. Figure 2 shows the Testing tab for the develop interface of the io.picolabs.wovyn.router ruleset in the pico named sensor_line after the lastTemperature query has been made. Because this is a live view into the running system, the interface can be used to query the state and raise events in the pico.

Figure 2: Interacting with a Pico (click to enlarge)

A pico's state is updated by rules running in the pico in response to events that the pico sees. Pico state is made available to rules as persistent variables in KRL, the ruleset programming language. When a rule sets a persistent variable, the state is persisted after the rule has finished execution and is available to other rules that execute later2. The Testing tab allows developers to raise events and then see how that impacts the persistent state of the pico.

Programming Picos

As I said, when I saw r0ml's talk, I was immediately struck by how much programming picos felt like using the Smalltalk or Lisp image. In some ways, it's like working with Docker images in a Fargate-like environment since it's serverless (from the programmer's perspective). But there's far less to configure and set up. Or maybe, more accurately, the setup is linguistically integrated with the application itself and feels less onerous and disconnected.

Building a system of picos to solve some particular problem isn't exactly like using Smalltalk. In particular, in a nod to modern development methodologies, the rulesets are installed from URLs and thus can be developed in the IDE the developer chooses and versioned in git or some other versioning system. Rulesets can be installed and managed programmatically so that the system can be programmed to manage its own configuration. To that point, all of the interactions in developer interface are communicated to the pico via an API installed in the picos. Consequently, everything the developer interface does can be done programmatically as well.

Figure 3 shows the programming workflow that we use to build production pico systems.

Figure 3: Programming Workflow (click to enlarge)

The developer may go through multiple iterations of the Develop, Build, Deploy, Test phases before releasing the code for production use. What is not captured in this diagram is the interactive feel that the pico engine provides for the testing phase. While automated tests can test the unit and system functionality of the rules running in the pico, the developer interface provides a visual tool for envisioning the interaction of the picos that are animated by dynamic interactions. Being able to query the state of the picos and see their reaction to specific events in various configurations is very helpful in debugging problems.

A pico engine can use multiple images (one at a time). And an image can be zipped up and shared with another developer or checked into git. By default the pico engine stores the state of the engine, including the installed rulesets, in the ~/.pico-engine/ directory. This is the image. Developers can change this to a different directory by setting the PICO_ENGINE_HOME environment variable. By changing the PICO_ENGINE_HOME environment variable, you can keep different development environments or projects separate from each other, and easily go back to the place you left off in a particular pico application.

For example, you could have a different pico engine image for a game project and an IoT project and start up the pico engine in either environment like so:

# work on my game project PICO_ENGINE_HOME=~/.dnd_game_image pico-engine # work on IoT project PICO_ENGINE_HOME=~/.iot_image pico-engine Images and Modern Development

At first, the idea of using an image or the running system and interacting with it to develop an application may see odd or out of step with modern development practices. After all, developers have had the idea of layered architectures and separation of concerns hammered into them. And image-based development in picos seems to fly in the face of those conventions. But it's really not all that different.

First, large pico applications are not generally built up by hand and then pushed into production. Rather, the developers in a pico-based programming project create a system that comes into being programmatically. So, the production image is separate from the developer's work image, as one would like. Another way to think about this, if you're familiar with systems like Smalltalk and Lisp is that programmers don't develop systems using a REPL (read-eval-print loop). Rather they write code, install it, and raise events to cause the system to talk action.

Second, the integration of persistence into the application isn't all that unusual when one considers the recent move to microservices, with local persistence stores. I built a production connected-car service called Fuse using picos some years ago. Fuse had a microservice architecture even though it was built with picos and programmed with rules.

Third, programming in image-based systems requires persistence maintenance and migration work, just like any other architecture does. For example, a service for healing API subscriptions in Fuse was also useful when new features, requiring new APIs, were introduced since the healing worked as well for new, as it did existing, API subscriptions. These kinds of rules allowed the production state to migrate incrementally as bugs were fixed and features added.

Image-based programming in picos can be done with all the same care and concern for persistence management and loose coupling as in any other architecture. The difference is that developers and system operators (these days often one and the same) in a pico-based development activity are saved the effort of architecting, configuring, and operating the persistence layer as a separate system. Linguistically incorporating persistence in the rules provides for more flexible use of persistence with less management overhead.

Stored procedures will not likely soon lose their stigma. Smalltalk images, as they were used in the 1980's, are unlikely to find a home in modern software development practices. Nevertheless, picos show that image-based development can be done in a manner consistent with the best practices we use today without losing the important benefits it brings.

Future Work

There are some improvements that should be made to the pico-engine to make image-based development better.

Moving picos between engines is necessary to support scaling of pico-based system. It is still too hard to migrate picos from one engine to another. And when you do, the parent-child hierarchy is not maintained across engines. This is a particular problem with systems of picos that have varied ownership. Managing images using environment variables is clunky. The engine could have better support for naming, creating, switching, and deleting images to support multiple project. Bruce Conrad has created a command-line debugging tool that allows declarations (which don't affect state) to be evaluated in the context of a particular pico. This needs functionality could be better integrated into the developer interface.

If you're intrigued and want to get started with picos, there's a Quickstart along with a series of lessons. If you want support, contact me and we'll get you added to the Picolabs Slack.

The pico engine is an open source project licensed under a liberal MIT license. You can see current issues for the pico engine here. Details about contributing are in the repository's README.

Notes These properties are dependent on the underlying pico engine and the persistence of picos is subject to availability and correct operation of the underlying infrastructure. Persistent variables are lexically scoped to a specific ruleset to create a closure over the variables. But this state can be accessed programmatically by other rulesets installed in the same pico by using the KRL module facility.

Tags: picos identity persistence programming

Sunday, 07. February 2021

John Philpin : Lifestream

”A long time ago my friend Doc Searls and I agreed that yo

”A long time ago my friend Doc Searls and I agreed that you don’t make money from a blog or podcast, but you can make money because of a blog or podcast. It’s a subtle but important difference.” Dave Winer

”A long time ago my friend Doc Searls and I agreed that you don’t make money from a blog or podcast, but you can make money because of a blog or podcast. It’s a subtle but important difference.”

Dave Winer


Doc Searls Weblog

Why the Chiefs will win the Super Bowl

I think there are more reasons to believe in the Bucs than the Chiefs today: better offensive line, better defense, Brady’s unequaled Super Bowl experience, etc. But the Chiefs are favored by 3.5 points, last I looked, and they have other advantages, including the best quarterback in the game—or maybe ever—in Patrick Mahomes. And that’s […]

I think there are more reasons to believe in the Bucs than the Chiefs today: better offensive line, better defense, Brady’s unequaled Super Bowl experience, etc. But the Chiefs are favored by 3.5 points, last I looked, and they have other advantages, including the best quarterback in the game—or maybe ever—in Patrick Mahomes.

And that’s the story. The incumbent GOAT (greatest of all time) is on his way out and the new one is on his way in. This game will certify that. I also think the Chiefs will beat the spread. By a lot. Because Mahomes and the Chiefs’ offense is just that good, and that ready.

Disclosures… In 2016, I correctly predicted, for the same reason (it makes the best story) that Lebron James and the Cleveland Cavaliers would beat the Golden State Warriors for the NBA championship. Also, a cousin of mine (once removed—he’s the son of my cousin) is Andy Heck, the Chiefs’ offensive line coach. So, as a long-time fan of both the Patriots and Tom Brady, I’ll be be cool with either team winning.

But I do think a Chiefs win makes a better story. Especially if Mahomes does his magic behind an offensive line of injuries and substitutes outperforming expectations.

[Later…] The Chiefs lost, 31-9, and their o-line was terrible. Poor Pat had to use his scrambling skills to the max, running all over the backfield looking for a well-covered receiver. And he came inches from hitting one in the end zone at least twice, while on the run 50 or more yards away. This was the Chief’s worst loss ever in the Mahomes era. Anyway, it looked and felt like it. But hey: congrats to the Bucs. They truly kicked ass.

 

 


Simon Willison

Video introduction to Datasette and sqlite-utils

I put together a 17 minute video introduction to Datasette and sqlite-utils for FOSDEM 2021, showing how you can use Datasette to explore data, and demonstrating using the sqlite-utils command-line tool to convert a CSV file into a SQLite database, and then publish it using datasette publish. Here's the video, plus annotated screen captures with further links and commentary.

I put together a 17 minute video introduction to Datasette and sqlite-utils for FOSDEM 2021, showing how you can use Datasette to explore data, and demonstrating using the sqlite-utils command-line tool to convert a CSV file into a SQLite database, and then publish it using datasette publish. Here's the video, plus annotated screen captures with further links and commentary.

Datasette is an "open source multi-tool for exploring and publishing data". The best way to explain what that means it to show you a demo.

Cleo is my dog. I'm going to answer the question "What is Cleo's favourite coffee shop?"

I use the Foursquare Swarm app. Every time I check in somewhere with Cleo I use the wolf emoji as part of my checkin message.

I use my swarm-to-sqlite tool to build a database of all of my checkins using data retrieved from the Foursquare API.

This is a SQL view that joins against the venues table to retrieve the latitude and longitude, then uses the datasette-cluster-map plugin to show my checkins on a map.

The view is defined by the following SQL:

CREATE VIEW checkin_details AS select checkins.id, created, venues.id as venue_id, venues.name as venue_name, venues.latitude, venues.longitude, group_concat(categories.name) as venue_categories, shout, createdBy, events.name as event_name from checkins join venues on checkins.venue = venues.id left join events on checkins.event = events.id join categories_venues on venues.id = categories_venues.venues_id join categories on categories.id = categories_venues.categories_id group by checkins.id order by createdAt desc;

If I filter for everything where the "shout" contains the wolf emoji I get a map of just the places that Cleo likes.

If I facet by venue_categories I can see that she likes parks and dog runs, but she's also been to coffee shops 25 times.

I can filter down to just coffee shop checkins, then facet by venue_name to see that she's been to Blue Bottle 16 times but she has occasionally been to Starbucks.

Everything you can see in the Datasette interface can be got back out as raw data as well. This is a JSON feed of Cleo's coffee shop checkins.

And here are those checkins as CSV.

So this is clearly super-useful software for categorizing your dog's different coffee shop visits.

Let's try something a bit more serious.

The New York Times publish their Covid-19 data to a GitHub repository as CSV files on a daily basis.

I run a Datasette instance at covid-19.datasettes.com which publishes the latest COVID data from the New York Times, the LA Times and some other sources on a daily basis.

Here are the New York Times daily county numbers as a table in Datasette.

I can drill down to just the numbers for San Francisco County in California.

Then I can use the datasette-vega plugin to plot those numbers on a line chart.

Here's that same chart datasette-vega plugin to for Los Angeles county.

Since this is all built on top of a relational database, I can use joins. Here's a page that joins this the New York times data aginst this table of US Census county populations by FIPS code, allowing us to see number of cases per million population.

Another plugin is datasette-copyable, which gives you the ability to copy and paste data out of Datasette in different formats - here's a LaTeX export of that COVID data for example.

Datasette is built on top of SQLite, which claims to be the world's Most Widely Deployed and Used Database Engine.

I particularly like how SQLite databases are single .db files on disk, which makes them easy to copy, upload and share with other people.

The original inspiration for Datasette was work I did at the Guardian back in 2009. We launched the Datablog to share the data underlying our stories - and we ended up using Google Sheets to share the data. I always felt like there should be a more open way of publishing data than putting it in a Google Sheet.

The other inspiration was serverless hosting providers, like Vercel and Cloud Run.

I started looking at these three years ago, and realized that while they didn't generally provide a relational database (at least not for free) if your data was read-only you could bundle the data up in a SQLite file and deploy it as part of the application code.

I call this the Baked Data architectural pattern.

The city of San Francisco publishes a CSV file of every tree in the city - 195,000 trees!

The CSV file includes latitude, longitude, species, street address and lots more.

sqlite-utils is a separate tool I've been building for manipulating SQLite databases.

I'm going to use it to insert that tree CSV file into a SQLite database file.

sqlite-utils insert trees.db trees Street_Tree_List.csv --csv

Full documentation for the sqlite-utils insert command is available here.

Both Datasette and sqlite-utils can be installed on macOS using Homebrew.

brew install datasette sqlite-utils

They are written in Python so they can also be installed using pip.

To run Datasette against the new trees.db file locally, run this:

datasette trees.db

Then visit http://127.0.0.1:8000/ in your browser.

You can also use the -o option to Datasette to open the browser for you:

datasette trees.db -o

And here's that CSV file running in Datasette in tabular format.

To see them on a map, we need the datasette-cluster-map plugin. We can install that like this:

datasette install datasette-cluster-map

And here they are on a map. You can see straight away that sixteen of the trees have an incorrect latitude and longitude and appear in the middle of the ocean.

A lot of the columns have duplicate values - qLegalStatus for example. sqlite-utils has a tool for helping with that - it can extract those values out into a separate table.

I wrote about this at length in Refactoring databases with sqlite-utils extract.

We can extract that qLegalStatus column like so:

sqlite-utils extract trees.db trees qLegalStatus

When I refresh the page you can see that those columns now contain links through to the new table.

Here's the new table.

And here's a map of every "significant tree" in San Francisco.

We'll extract two more columns:

sqlite-utils extract trees.db trees qCaretaker sqlite-utils extract trees.db trees qSpecies

Another useful thing we can do is configure full-text search. We can do that against the qAddress column using the enable-fts command:

sqlite-utils enable-fts trees.db trees qAddress

Now Datasette provides a search field, which we can use to search for "grove" to see a map of all of the trees on Grove street.

And I can facet by species to see the most common species of trees on Grove street. I can even get those back out as CSV or JSON.

Because this is all running on a SQLite database, you can execute custom SQL queries against the database. This is safe because the database is opened in read-only mode and there's a time limit on how long a query is allowed to execute for.

This means you can write applications in JavaScript that execute SQL against Datasette and pull back the results in JSON. I wrote more about this in a tutorial called Fast Autocomplete Search for Your Website.

Let's publish this to the internet, using the datasette publish command.

I'm going to publish trees.db to Google Cloud Run using the following commmand:

datasette publish cloudrun trees.db --title "San Francisco Trees" --service sf-trees

This will create a new Docker container with Datasette and the bundled trees database, push that to Google's cloud infrastructure, and Google will then deploy it to Cloud Run and provide a URL.

The Datasette website at datasette.io has details of both Datasette and the many other pieces that make up the overall Datasette project.

There are new 56 plugins in the plugin directory there.

There's also a collection of tools for working with SQLite databases - both manipulating existing databases and creating new ones with data pulled from APIs, different file formats and other relational databases.

Now thet the publish flow has finished, you can interact with the trees database online at sf-trees-j7hipcg4aq-uc.a.run.app.

To deploy it using the datasette-cluster-map plugin, run this:

datasette publish cloudrun trees.db --title "San Francisco Trees" \ --service sf-trees --install=datasette-cluster-map

The official Datasette website is itself an instance of Datasette. It uses custom templates for the homepage and other pages.

You can browse the underlying data tables at datasette.io/content.

It also has a search engine (built using dogsheep-beta) which can search across plugins, releases, documentation, blog entries and more.

Here's an example search for CSS.

If you'd like to have a one-on-one conversation with me about Datasette you can sign up for Datasette office hours. I love talking to people about these projects!


Jon Udell

Continental drift

In a 1999 interview David Bowie said that “the potential for what the Internet is going to do to society, both good and bad, is unimaginable.” I had no problem imagining the good, it was the bad where my imagination failed. The web of the late 90s was a cornucopia of wonder, delight, and inspiration. … Continue reading Continental drift

In a 1999 interview David Bowie said that “the potential for what the Internet is going to do to society, both good and bad, is unimaginable.” I had no problem imagining the good, it was the bad where my imagination failed. The web of the late 90s was a cornucopia of wonder, delight, and inspiration. So was the blogosophere of the early 2000s. I know a lot of us are nostalgic for those eras, and depressed about how things have turned out. The bad is really quite bad, and sometimes I feel like there’s no way forward.

And then something wonderful happens. This time the spark was David Grinspoon aka @DrFunkySpoon. I’ve written before about a Long Now talk in which he posits that we might not just be living through the beginning of a geological epoch called the Anthropocene but rather — and far more profoundly — the dawn of an eon that he calls the Sapiezoic. Today he posted a stunning new visualization of plate tectonics.

As always when I think about plate tectonics, I’m reminded of the high school science teacher who introduced me to the topic. His name is John Ousey, and this happened almost 50 years ago. What always stuck with me is the way he presented it. Back then, plate tectonics was a new idea. As Mr. Ousey (he later became Dr. Ousey) described the continents sliding apart, I can still see the bemused look on his face. He was clearly wrestling with the concept, unsure whether to believe things really happen that way. That healthy skepticism, coupled with trust in the scientific process, made an indelible impression on me.

One of the wonders of the Internet is the ability to find people. It took some sleuthing, but I did find him and this ensued.

I wrote to John Ousey and he replied!

"I learned about Plate Tectonics (then called Continental Drift) by taking weekend courses taught by the great teacher/researcher Ehrling Dorf of Princeton. It was brand new and I'm not sure that he was totally convinced about it either."

— Jon Udell (@judell) February 6, 2021

That’s the kind of magic that can still happen, that does happen all the time.

I learned for the first time that John Ousey’s introduction to plate tectonics came by way of “weekend courses taught by the great teacher/researcher Ehrling Dorf of Princeton” who was himself perhaps “not totally convinced.” Despite uncertainty, which he acknowledged, John Ousey was happy to share an important new idea with his Earth science class.

What a privilege to be able to thank him, after all these years, for being a great teacher who helped me form a scientific sensibility that has never mattered more than now. And to share a moment of appreciation for an extraordinary new visualization of the process once known as continental drift. Yes, there’s a dark side to our connected world, darker than I was once willing to imagine. But there is also so much light. It’s always helpful to consider deep geological time. That video shows a billion years of planetary churn. We’ve only been connected like this for 25 years. Maybe we’ll figure it out. For today, at least, I choose to believe that we will.


Simon Willison

Weeknotes: Mostly messing around with map tiles

Most of what I worked on this week was covered in Serving map tiles from SQLite with MBTiles and datasette-tiles. I built two new plugins: datasette-tiles for serving map tiles, and datasette-basemap which bundles map tiles for zoom levels 0-6 of OpenStreetMap. I also released download-tiles for downloading tiles and bundling them into an MBTiles database. sqlite-utils 3.4.1 I added one new fe

Most of what I worked on this week was covered in Serving map tiles from SQLite with MBTiles and datasette-tiles. I built two new plugins: datasette-tiles for serving map tiles, and datasette-basemap which bundles map tiles for zoom levels 0-6 of OpenStreetMap. I also released download-tiles for downloading tiles and bundling them into an MBTiles database.

sqlite-utils 3.4.1

I added one new feature to sqlite-utils: the sqlite-utils import command can now be configured to read CSV files using alternative delimiters, by passing the --delimiter option or the --quotechar option.

This is covered in the documentation, which provides the following example:

name;description Cleo;|Very fine; a friendly dog| Pancakes;A local corgi

Imported using:

sqlite-utils insert dogs.db dogs dogs.csv \ --delimiter=";" --quotechar="|" Datasette 0.54.1

I spotted a subtle but nasty regression in Datasette: a change I made to how hidden form fields worked on the table page meant that clearing the _search search input and re-submitting the form didn't take effect, and the search would persist. Datasette 0.54.1 fixes that bug.

Releases this week datasette-jellyfish: 1.0.1 - 2021-02-06
Datasette plugin adding SQL functions for fuzzy text matching powered by Jellyfish sqlite-utils: 3.4.1 - 2021-02-06
Python CLI utility and library for manipulating SQLite databases datasette-tiles: 0.5 - 2021-02-04
Mapping tile server for Datasette, serving tiles from MBTiles packages download-tiles: 0.4 - 2021-02-03
Download map tiles and store them in an MBTiles database datasette-basemap: 0.2 - 2021-02-02
A basemap for Datasette and datasette-leaflet datasette: 0.54.1 - 2021-02-02
An open source multi-tool for exploring and publishing data datasette-cluster-map: 0.17.1 - 2021-02-01
Datasette plugin that shows a map for any data with latitude/longitude columns datasette-leaflet: 0.2.2 - 2021-02-01
Datasette plugin adding the Leaflet JavaScript library TIL this week Splitting on commas in SQLite Serving MBTiles with datasette-media Downloading MapZen elevation tiles

Saturday, 06. February 2021

John Philpin : Lifestream

'

”Joe Biden Makes Shocking Announcement: None of His Children or Grandchildren Will Be Named Senior Advisers to the President.” Vanity Fair</:>

”Joe Biden Makes Shocking Announcement: None of His Children or Grandchildren Will Be Named Senior Advisers to the President.”

Vanity Fair</:>


I THINK I have shared this before - but better to see it twi

I THINK I have shared this before - but better to see it twice than not at all - right?

I THINK I have shared this before - but better to see it twice than not at all - right?

Friday, 05. February 2021

Identity Woman

Radical Exchange Talk: Data Agency. Individual or Shared?

I had a great time on this Radical Exchange conversation The post Radical Exchange Talk: Data Agency. Individual or Shared? appeared first on Identity Woman.

I had a great time on this Radical Exchange conversation

The post Radical Exchange Talk: Data Agency. Individual or Shared? appeared first on Identity Woman.


John Philpin : Lifestream

”Zuckerberg is a pretty shallow thinker on anything outsid

”Zuckerberg is a pretty shallow thinker on anything outside of Facebook.” Steve Schmidt I agree, just happy to finally hear - and hear I did - someone say it.

”Zuckerberg is a pretty shallow thinker on anything outside of Facebook.”

Steve Schmidt

I agree, just happy to finally hear - and hear I did - someone say it.

Thursday, 04. February 2021

MyDigitalFootprint

Quantum Risk: a wicked problem that emerges at the boundaries of our data dependency

Framing the problem I am fighting bias and prejudice about risk perceptions; please read the next lines before you click off.  We tend to be blind sighted to “risk” because we have all lived it, read it and listened to risk statements.  The ones on the TV and radio for financial products, the ones at the beginning of investment statements, ones for health and safety for machinery, ones
Framing the problem

I am fighting bias and prejudice about risk perceptions; please read the next lines before you click off.  We tend to be blind sighted to “risk” because we have all lived it, read it and listened to risk statements.  The ones on the TV and radio for financial products, the ones at the beginning of investment statements, ones for health and safety for machinery, ones for medicine, ones on the packets of cigarettes, the one when you open that new app on your new mobile device. We are bombarded with endless risk statements that we assume we know the details of, or just ignore.  There are more books on risk than on all other management and economics topics together.  There is an entire field on the ontologies of risk; such is the significance of this field. This article is suggesting that all that body of knowledge and expertise has missed something.  A bold statement, but quantum risk is new, big, ugly, and already here, it's just that we are willingly blind to it. 

At the end of the Board pack or PowerPoint deck for new investment, intervention case or for the adoption of the new model, there is a risk and assumptions list.  We have seen these so many times we don’t read them.  These statements are often copies, and the plagiarism of risk statements inaccurately copied is significant; no effort is put in as such statements have become a habit in the process methodology.   The problem we all have with risk is that we know it all. Quite frankly, we see risk as the prime reason to stop something and occasionally manage it closer but never too understand something better.  If you are operating a digital or data business you have new risks that are not in your risk statement, you have not focussed on them before, you are unlikely to have been exposed to them, and this article is to bring them to your attention.  Is that worth 8 minutes?

Many thanks to Peadar Duffy whom I have been collaborating with on this thinking, and he has published a super article on the same topic (quantum risk) here 

The purpose of business 

We know that 3% of our data lake is finance data today; shockingly, 90% of our decisions are based on this sliver of data (Source Google). As we have to aim for a better ratio of “data: decisions” that includes non-financial data; we will make progress towards making better decisions that benefit more than a pure shareholder primacy view of the world.  As leaders, we have a desire to make the best possible decisions we can. We fuse data, experience and knowledge to balance our perception of risk, probability and desired outcomes.  

The well-publicised “Business Roundtable” report in Aug 2019 redefines a corporation’s purpose to promote ‘An Economy That Serves All … [Americans]’.  The idea that company purpose should be closer to ecosystem thinking has been gaining prevalence since the financial crisis in 2008.  The thinking has significant supporters such as Larry Fink,  Blackrock’s founder and CEO, who is an influential voice for ESG reporting and promotes changes to decision making criteria for better outcomes. His yearly letters are an insightful journey. 

Sir Donald Brydon's Dec 2019 report highlights that governance and audit need attention if we are to deliver better decisions, transparency and accountability. The report concludes that audit has significant failings and our approach to tick box compliance is not serving directors, shareholders or society to the level expected. Given that so much of our risk management depends on the quality of the audit, internal and external, it is likely that we are unduly confident in data that is unreliable. This point alone about audit failure could be sufficient for this article’s conclusion; however, we are here to explore Quantum Risk. Quantum Risk only exists because of the business dependency we now have on data from our co-dependent supply chains to dependent ecosystems.  

Quantum Risk is NEW  

As a term from physics that describes particles’ properties, “quantum” will help frame new risk characteristics.  The primary characteristics of quantum particles’ behaviour are:- the uncertainty principle, composite systems and entanglement.   In a language, I understand these characteristics for Quantum risk are:

When you observe the same risk twice, it might not be there, and it will look different.

The same risk can be in many places simultaneously, but it is only one risk.

Your risk and my risk directly affect each other across our data ecosystem; they are coupled but may not be directly connected.

Framing Risk

Risk, like beauty, privacy, trust, transparency and many other ideals, is a personal perspective on the world; however, we all accept that we have to live with risk.

Risk, and the management of risk, fundamentally assumes that you can identify it first.  If you cannot identify the risk, there is no risk to consider or manage. 

Having identified the risk, you assess the risks to categorise and prioritise them using the classic impact vs likelihood model. 

Finally, the management (review and control) of risk determines if you are doing the right things or action is needed.  

It is possible to add a third axis to a classic likelihood, impact risk model, “quality of knowledge.” The third axis visually highlights that a focus on high risks accumulates the most knowledge as that is where the management focus and control is required, and it needs data which becomes knowledge.    If there is a deficit in knowledge because of poor data, it translates into an increased risk hidden because of poor data at any point in the matrix.  Poor data (knowledge) can mean that either the impact (consequence) will be more severe or the likelihood (probability) is more likely. In part, we can overcome poor data problems by recognising that it always exists, but it easily hides the rather current issues of pandemics and systemic risk. However, if the quality of knowledge is based on erroneous data (data without rights and attestation), we have no truth to the likelihood and impact.

 


Some sophisticated models and maths help qualify and understand the nature of risk depending on its nature and size.  However, the list of risks that any one company faces is defined, specified and has been thought about over a long period.  Uncovering new risk is considered unlikely; however, it is this that we are exploring and given our natural confirmational bias towards risk (we know it) - this is hard.  

Classic risk models are framed to gain certainty, where risk is the identification, understanding, management and control of uncertainty.  Existing risk models are highly efficient within this frame of reference, and we can optimise to our agreed boundaries of control with incredible success.  Risk within our boundary (sphere of direct control) is calculated, and it becomes a quantified measure, enabling incentives to be created that provide a mechanism for management control.   Risk outside our boundary (indirect control on a longer supply or value chain), whilst it is someone else’s risk we are dependent on them to manage it. Such dependencies are vital in modern global businesses. We have developed methodology (contracts) and processes (audit) to ensure that we are confident that any risk to us, inside or outside of our direct control, is identified and managed.

However, as leaders, we face three fundamental issues on this move to an economy that serves broader eco-systems as the boundaries we are dependent on have become less clear.  

1. The quality of the data and implied knowledge we receive from our direct and dependent* eco-system, even if based on audit for financial and non-financial data, is unreliable and is increasingly complicated due to different data proposes and ontologies.

2. The quality of the knowledge we receive from our indirect and interdependent** eco-system, even if based on audit for financial and non-financial data, is unreliable and is increasingly complicated due to different data proposes and ontologies.

3. Who is responsible and accountable at second and third-order data boundaries? (assumption first boundary is direct and already in control in our risk model)

* Dependent: balancing being able to get what I want by my own effort as contingent on or determined by the actions of someone else to make it work  ** Interdependence combine my efforts with the efforts of others to achieve successful outcomes together but does not have to be mutual or controlled 

Risk as a shared belief has wider dependencies. 

Who is responsible and accountable at second and third-order data boundaries? (Point 3 above) introduces the concept of second and third-order boundaries for broader (inter)-dependent ecosystems. This short section explains where those boundaries are and why they matter in the context of a business’s purpose moving toward a sustainable ecosystem (ESG.)

The figure below expands on the dependency thinking into a visual representation. The three-axis are values/ principles as a focus [self, society, planet earth], who has accountability/ obligations [no-one, an elected authority such as a director, society or all of humanity], and the health of our eco-systems (prime, secondary, tertiary and all).

The small blue area shows the limitations of our current shareholder primacy remit, where Directors have a fiduciary duty to ensure that their prime business thrives and value is created for shareholders (stakeholders,) at the expense of others. Having a healthy ecosystem helps (competition, choice, diverse risk, margin.)  As envisaged by the Business Roundtable, a sustainable ecosystem is the orange area, expanding the Directors remit to more eco-systems and embracing more of a “good for society” value set but does not increase director accountability.  ESG v1.0 widens the remit to the green area; this step-change expands all current thinking and dependencies of any one player on others on a broader ecosystem. We become sustainable together. 

How is it possible for unidentified risks to exist?

In simple terms, there is no new unknown risk; however, what is known to someone may not be known by everyone. Risk is hiding in plain sight. As we are expanding our remits as discussed in the last section above, we are increasingly dependent on others managing their risk to the same level we manage risk and share data across the ecosystem. This is where Quantum Risk arises, at the boundaries, in the long-tail of the universe of risk.

In the figure below, The Growing Universe of Risk. We are very good at the management of insurable, measurable known:known (identified and shared) risk. We are also very good at un-insurable, measurable (impact, likelihood, knowledge) and known:unknown risk mainly because the determined likelihood of occurrence and impact is moderate.  Indeed, we have created excellent tools to help mitigate and accept uninsurable, un-measurable “unknown:unknown” risk.  In mitigation we accept that the data quality  (knowledge) is poor, but the impact is low, as is the likelihood.  

Quantum risk is the next step out; it is emergent at the boundaries of (inter)-dependencies created as we need to create sustainable ecosystems where we share data. We are increasingly reliant on data from indirectly related players to our ecosystem, and we have no power or control. We have no rights to data and no clue on attestation. Quantum risk is not in our current risk model, or existing risk frameworks and is unimagined to us. 


Business Risk Vs Data Risk

Business risk is something that every business has to deal with.  Kodak and Nokia maybe not as well as say IBM, Barclays or Microsoft.   Mobile phone networks should have seen mobile data services coming and therefore the advent of international voice and video apps that meant there was always going to be a natural decline in SMS, local and international mobile revenue. Most rejected this business risk in 2005 only seeing growth in core areas.  However good hindsight is, apps such as Signal, WhatsApp and Telegram came about due to the timing of three interrelated advances, which created business risk.   Device capability, network capability and pricing.  Device designers and manufacturers have to keep pushing technology to keep selling devices; device technology will always advance.   Network capacity was always going to increase, and packet-switched capability has massive economies of scale over voice circuits. Large, fast packet circuits were always going to win.  Pricing by usage prevents usage; bundles work for increasing capacity.  For a mobile operator, the objective is to fill the network capacity that is built to maximise ROI, bundles work, as does Apps that move revenue from one product to the next.  This is a business risk created by change and dependencies on others in your ecosystem, quantum risks are a business risk but hide in data.

Data Risk falls into three buckets.   

Data that you collect directly as part of the process of doing business.  Critically you can determine the attestation (provenance and lineage) of the data, and it comes from your devices, sensors and systems.  There is a risk that you don’t collect, store, protect, analyse or know if the data is true.  In truth, this is the front end of the long tail in the universe of risk, and it is precisely where we put priority. Nothing new here.

Data you collect from third parties who you have a relationship with. A supplier, partners, collaborator, associate or public data.  Whilst you are likely to have the “rights to use data” (contract), you are unlikely to have “attestation” (provenance and lineage) of the shared data back to the origin. You will have access to summary or management levels (knowledge and insights), and you should have audit and other contractual agreements to check.   There is often a mutual relationship where you both share data, both dependent on the data quality. The risk is that you don’t qualify, check, question or analyse this 3rd party data.  In truth, this is another head-end risk of the long tail in the universe of risk, and it is precisely where we put significant resources. The exception will be public data as there is no route to understanding bias, ontology or purpose, however public data is not usually used exclusively for decision making, with one exception right now ESG and this worries me.  

Quantum Risk is a data risk where you neither have control of nor access to, data. Still, this data set has become critical to decision making as we move to sustainable ecosystems, stewardship codes and ESG.  However, it requires us to dig into the dark and mysterious world data ontologies, which we have to unpack quickly.   

Ontologies  

To explain your reasoning, rationale or position, you need to define how entities are grouped into basic categories that structure your worldview and perspective. If you have a different perspective, you will behave and act differently.  Such a structure is called ontology (how we view the world) and is related to epistemology (how do we know what is true and how we have gone about investigating/ proving it?). Ontology is a branch of philosophy but is critical in understanding data and information science as it encompasses a representation, formal naming and definition of the categories, properties and relations between the concepts, data and entities that substantiate one, many, or all domains of discourse. Think of data ontology as a way of showing the properties of a subject area and how they are related, by defining a set of concepts and categories that represent the subject. 

At this point you would have thought with 5,000 years of thinking about this we would have one top-level ontology from which everything would flow.  Alas, we don’t have one for anything.  There is no black and white agreed way to look at anything in philosophy, physics, biology, humanities, data, climate, language, sound, knowledge, compute, behaviour and every other topic. This means that it is safe to assume your way of describing your world, in your organisation, through data is different from everyone else in your ecosystem.  Those same data points represented in 1 and 0’s mean completely other things in different ontologies. Your worst scenario is different ontologies inside your silos which means you have different world views but may not know this.  Ontology is one of the roles for a CDO, explored here.  Now to epistemology, which is concerned with the creation of knowledge, focusing on how knowledge is obtained and investigating the most valid ways to reach the truth. Epistemology essentially determines the relationship between the data, analyst and reality and is rooted in your ontological framework. Different data science teams can have the same data set and very different views, and then we add the statistics team.  What truth or lies do you want?  This matters when data is shared - how do you know what your business partners thinks is true about their data?

It only gets more complicated the more you unpack this and I will write an article about this soon. However, as shown in the figure, knowing how you view the world in data, does not guarantee that everyone else in your ecosystem has the same view.  I have seen very few contracts for data sharing at business data levels share the ontology and mapping schedules between then. Yes we often share naming/ data dictionary level, but that is not ontology. Assuming that shared data has the same purpose between the partner is “quantum risk.” This risk is at the boundaries, and it only appears when you look.  Imagine you are sharing data in your ecosystem on critical systems and as you read this, you realise you have not asked the question about the different world views you and your partners have towards collecting, analysing, and reporting for data.  The event is not the same thing.  Remember, at the start, we know everything about risk. I am in the same bucket. This is all new.  

Responses to Quantum Risk

I made two bold claims at the beginning. “The problem we all have with risk is that we know it all,” and “a bold statement, but quantum risk is new, big, ugly and is already here, it's just that we are willingly blind to it.”  I wish it were easy, but Quantum Risk emerges at our digital business boundaries where we share data, the further we go out the less attestation and rights we have. The complexity of Quantum Risk creates havoc with our existing frameworks and models as:

When you observe the same quantum risk twice, it might not be there, and it will look different.

The same quantum risk can be in many places at the same time, but it is only one risk.

Your quantum risk and my quantum risk directly affect each other across our data ecosystem, but they are not connected and not seen.

Given this, how do we respond? We need to get better with understanding the purpose of our data; we need to find CDO expertise to help us unpack our data ontologies and rethink what we consider are boundaries for commercial purposes, which means revisiting our contracts and terms.  One question for those who get this far, have you tested how your users understand your Terms and Conditions on data use and privacy. I have never seen it in a test schedule as it is a barrier not a value proposition. We tell users to “Click here” fast and trust us. It is an obvious gap to investigate from a partner when you depend on that data and it is shared with you, and your advertising model now depends on it.

Any good economist/ strategist will immediately recognise the opportunity to game data at the boundary. How can I create an advantage, and what are the implications is another whole topic to unpack.  

As a final thought, will your corporation consider Quantum Risk? 

If your fellow senior leadership team is focused on the head end of the long tail, you will see a focus on implementing processes that align to regulation/ rules/ law and policies. You are likely to manage risk very well and be rewarded for doing so via cascading KPI’s.  Quantum risk will be thought about when there are best practices or a visible loss of competitive position.   

Corporates with a more mature risk profile know there are loopholes and whilst have a focus on compliance, they have a hand in the lobby forums so they can benefit by putting risk onto others and gaining an advantage from being the owner of IP when the lobby work becomes policy.  Quantum risk thinking will emerge when there is a clear identification of competitive advantage.

The most mature risk leadership teams are creating new thinking to ensure that they are sustainable and not forced to make retrospective changes as they just focussed on compliance and had delivery based KPI linked bonuses.  These are the pioneers in digital and will pick up quantum risk first. 



Phil Windley's Technometria

Announcing Pico Engine 1.0

Summary: I'm excited to announce a new, stable, production-ready pico engine. The latest release of the Pico Engine (1.X) provides a more modular design that better supports future enhancements and allows picos to be less dependent on a specific engine for operation. The pico engine creates and manages picos.1 Picos (persistent compute objects) are internet-first, persistent, actors

Summary: I'm excited to announce a new, stable, production-ready pico engine. The latest release of the Pico Engine (1.X) provides a more modular design that better supports future enhancements and allows picos to be less dependent on a specific engine for operation.

The pico engine creates and manages picos.1 Picos (persistent compute objects) are internet-first, persistent, actors that are a good choice for building reactive systems—especially in the Internet of Things.

Pico engine is the name we gave to the node.js rewrite of the Kynetx Rules Engine back in 2017. Matthew Wright and Bruce Conrad have been the principal developers of the pico engine.

The 2017 rewrite (Pico Engine 0.X) was a great success. When we started that project, I listed speed, internet-first, small deployment, and attribute-based event authorization as the goals. The 0.X rewrite achieved all of these. The new engine was small enough to be able to be deployed on Raspberry Pi's and other small computers and yet was significantly faster. One test we did on a 2015 13" Macbook Pro handled 44,504 events in over 8000 separate picos in 35 minutes and 19 seconds. The throughput was 21 events per second or 47.6 milliseconds per request.

This past year Matthew and Bruce reimplemented the pico engine with some significant improvements and architectural changes. We've released that as Pico Engine 1.X. This blog post discusses the improvements in Pico Engine 1.X, after a brief introduction of picos so you'll know why you should care.

Picos

Picos support an actor model of distributed computation. Picos have the following three properties. In response to a received message,

picos send messages to other picos—Picos respond to events and queries by running rules. Depending on the rules installed, a pico may raise events for itself or other picos. picos create other picos—Picos can create and delete other picos, resulting in a parent-child hierarchy of picos. picos change their internal state (which can affect their behavior when the next message received)—Each pico has a set of persistent variables that can only be affected by rules that run in response to events.

I describe picos and their API and programming model in more detail elsewhere. Event-driven systems, like those built from picos, can be used to create systems that meet the Reactive Manifesto.

Despite the parent-child hierarchy, picos can be arranged in a heterachical network for peer-to-peer communication and computation. As mentioned, picos support direct asynchronous messaging by sending events to other picos. Picos have an internal event bus for distributing those messages to rules installed in the pico. Rules in the pico are selected to run based on declarative event expressions. The pico matches events on the bus with event scenarios declared in the event expressions. Event expressions can specify simple single event matches, or complicated event relationships with temporal ordering. Rules whose event expressions match are scheduled for execution. Executing rules may raise additional events. More detail about the event loop and pico execution model are available elsewhere.

Each pico presents a unique Event-Query API that is dependent on the specific rulesets installed in the pico. Picos share nothing with other picos except through messages exchanged between them. Picos don't know and can't directly access or affect the internal state of another pico.

As a result of their design, picos exhibit the following important properties:

Lock-free concurrency—picos respond to messages without locks. Isolation—state changes in one pico cannot affect the state in other picos. Location transparency—picos can live on multiple hosts and so computation can be scaled easily and across network boundaries. Loose coupling—picos are only dependent on one another to the extent of their design. Pico Engine 1.0

Version 1.0 is a rewrite of pico-engine that introduces major improvements:

A more pico-centric architecture that makes picos less dependent on a particular engine. A more module design that supports future improvements and makes the engine code easier to maintain and understand. Ruleset versioning and sharing to facilitate decentralized code sharing. Better, attribute-based channel policies for more secure system architecture. A new UI written in React that uses the event-query APIs of the picos themselves to render.

One of our goals for future pico ecosystems is build not just distributed, but decentralized peer-to-peer systems. One of the features we'd very much like picos to have is the ability to move between engines seamlessly and with little friction. Pico engine 1.X better supports this roadmap.

Figure 1 shows a block diagram of the primary components. The new engine is built on top of two primary modules: pico-framework and select-when.

Figure 1: Pico Engine Modular Architecture (click to enlarge)

The pico-framework handles the building blocks of a Pico based system:

Pico lifecycle—picos exist from the time they're created until they're deleted. Pico parent/child relationships—Every pico, except for the root pico, has a parent. All picos may have children. Events—picos respond to events based on the rules that are installed in the pico. The pico-framework makes use of the select_when library to create rules that pattern match on event streams. Queries—picos can also respond to queries based on the rulesets that are installed in the pico. Channels—Events and queries arrive on channels that are created and deleted. Access control policies for events and queries on a particular channel are also managed by the pico-framework Rulesets—the framework manages installing, caching, flushing, and sandboxing rulesets. Persistence—all picos have persistence and can manage persistent data. The pico-framework uses Levelup to define an interface for a LevelDB compatible data store and uses it to handle persistence of picos.

The pico-framework is language agnostic. Pico-engine-core combines pico-framework with facilities for rendering KRL, the rule language used to program rulesets. KRL rulesets are compiled to Javascript for pico-framework. Pico-engine-core contains a registry (transparent to the user) that caches compiled rulesets that have been installed in picos. In addition, pico-engine-core includes a number of standard libraries for KRL. Rulesets are compiled to Javascript for execution. The Javascript produced by the rewrite is much more readable than that rendered by the 0.X engine. Because of the new modular design, rulesets written entirely in Javascript can be added to a pico system.

The pico engine combines the pico-engine-core with a LevelDB-compliant persistent store, an HTTP server, a log writer, and a ruleset loader for full functionality.

Wrangler

Wrangler is the pico operating system. Wrangler presents an event-query API for picos that supports programatically managing the pico lifecycle, channels and policies, and rulesets. Every pico created by the pico engine has Wrangler installed automatically to aid in programatically interacting with picos.

One of the goals of the new pico engine was to support picos moving between engines. Picos relied too heavily on direct interaction with the engine APIs in 0.X and thus were more tightly coupled to the engine than is necessary. The 1.0 engine minimizes the coupling to the largest extent possible. Wrangler, written in KRL, builds upon the core functionality provided by the engine to provide developers with an API for building pico systems programmatically. A great example of that is the Pico Engine Developer UI, discussed next.

Pico Engine Developer UI

Another significant change to the pico engine with the 1.0 release was a rewritten Developer UI. In 0.X, the UI was hard coded into the engine. The 1.X UI is a single page web application (SPA) written in React. The SPA uses an API that the engine provides to get the channel identifier (ECI) for the root pico in the engine. The UI SPA uses that ECI to connect to the API implemented by the io.picolabs.pico-engine-ui.krl ruleset (which is installed automatically in every pico).

Figure 2 shows the initial Developer UI screen. The display is the network of picos in the engine. Black lines represent parent-child relationships and form a tree with the root pico at the root. The pink lines are subscriptions between picos—two-way channels formed by exchanging ECIs. Subscriptions are used to form peer-to-peer (heterachical) relationships between picos and do no necessarily have to be on the same engine.

Figure 2: Pico Engine UI (click to enlarge)

When a box representing a pico in the Developer UI is clicked, the display shows an interface for performing actions on the pico as shown in Figure 3. The interface shows a number of tabs.

The About tab shows information about the pico, including its parent and children. The interface allows information about the pico to be changed and new children to be created. The Rulesets tab shows any rulesets installed in the pico, allows them to be flushed from the ruleset cache, and for new rulesets to be installed. The Channels tab is used to manage channels and channel policies. The Logging tab shows execution logs for the pico. The Testing tab provides an interface for exercising the event-query APIs that the rulesets installed in the pico provide. The Subscriptions tab provides an interface for managing the pico's subscriptions and creating new ones. Figure 3: Pico Developer Interface (click to enlarge)

Because the Developer UI is just using the APIs provided by the pico, everything it does (and more) can be done programatically by code running in the picos themselves. Most useful pico systems will be created and managed programmatically using Wrangler. The Developer UI provides a convenient console for exploring and testing during development. The io.picolabs.pico-engine-ui.krl ruleset can be replaced or augmented by another ruleset the developer installs on the pico to provide a different interface to the pico. Interesting pico-based system will have applications that interact with their APIs to present the user interface. For example, Manifold is a SPA written in React that creates a system of picos for use in IoT applications.

Come Contribute

The pico engine is an open source project licensed under a liberal MIT license. You can see current issues for the pico engine here. Details about contributing are in the repository's README.

In addition to the work on the engine itself, one of the primary workstreams at present is to complete Bruce Conrad's excellent work to use DIDs and DIDComm as the basis for inter-pico communication, called ACA-Pico (Aries Cloud Agent - Pico). We're holding monthly meetings and there's a repository of current work complete with issues. This work is important because it will replace the current subscriptions method of connecting heterarchies of picos with DIDComm. This has the obvious advantages of being more secure and aligned with an important emerging standard. More importantly, because DIDComm is protocological, this will support protocol-based interactions between picos, including credential exchange.

If you're intrigued and want to get started with picos, there's a Quickstart along with a series of lessons. If you want support, contact me and we'll get you added to the Picolabs Slack.

Notes The pico engine is to picos as the docker engine is to docker containers.

Photo Credit: Flowers Generative Art from dp792 (Pixabay)

Tags: picos iot krl programming rules


Jon Udell

How and why to tell your story online, revisited

I wrote this essay in 2006 as part of a series of Internet explainers I did for New Hampshire Public Radio. It never aired for reasons lost to history, so I’m publishing this 15-year-old time capsule here for the first time. My motive is of course not purely archival. I’m also reminding myself why I … Continue reading How and why to tell your story online, revisited

I wrote this essay in 2006 as part of a series of Internet explainers I did for New Hampshire Public Radio. It never aired for reasons lost to history, so I’m publishing this 15-year-old time capsule here for the first time. My motive is of course not purely archival. I’m also reminding myself why I should still practice now what I preached then.

How and why to tell your story online

Teens and twenty-somethings are flocking to social websites like MySpace and Facebook, where they post photos, music, and personal diaries. Parents, teachers, and cops wish they wouldn’t. It’s a culture war between generations, and right now everybody’s losing.

Kids: Listen up. Did you hear the story about the college student who didn’t get hired because of his Facebook page? Or the teenage girl whose MySpace blog told an attacker when she’d be home alone? These things happen very rarely, but they can happen. Realize that the words and pictures you publish online will follow you around for the rest of your lives. Realize that wrong choices can have embarrassing or even tragic consequences.

Now, grownups, it’s your turn to listen up. You’re right to worry about the kids. But there’s another side to the story. The new forms of Internet self-publishing — including social networks, blogs, podcasting, and video sharing — can be much more than narcissistic games. Properly understood and applied, they’re power tools for claiming identity, exerting influence, and managing reputation. Sadly, very few adults are learning those skills, and fewer still are teaching them.

It’s not enough to condemn bad online behavior. We’ve got to model good online behavior too — in schools, on the job, and in civic life. But we’re stuck in a Catch-22 situation. Kids, who intuit the benefits of the new social media, fail to appreciate the risks. Grownups, meanwhile, see only risks and no benefits.

There’s a middle ground here, and we need to approach it from both sides of the generation gap. The new reality is that, from now on, our lives will be documented online — perhaps by us, perhaps by others, perhaps by both. We may or may not influence what others will say about us. But we can surely control our own narratives, and shape them in ways that advance our personal, educational, professional, and civic agendas.

Your online identity is a lifelong asset. If you invest in it foolishly you’ll regret that. But failing to invest at all is equally foolish. The best strategy, as always, is to invest wisely.

Here’s a simple test to guide your strategy. Imagine someone searching Google for your name. That person might be a college admissions officer, a prospective employer, a new client, an old friend, or even a complete stranger. The reason for the search might be to evaluate your knowledge, interests, agenda, accomplishments, credentials, activities, or reputation.

What do you want that person to find? That’s what you should publish online.

To find three examples of what I mean, try searching the web for the following three names: Todd Suomela, Martha Burtis, Thomas Mahon. In each case, the first Google result points to a personal blog that narrates a professional life.

Todd Suomela is a graduate student at the University of Michigan. On his blog, Todd writes about what he’s learning, and about how his interests and goals are evolving. He hasn’t launched his professional career yet. But when he does, his habit of sharing the information resources he collects, and reflecting thoughtfully on his educational experience, will serve him well.

Martha Burtis is an instructional technologist at the University of Mary Washington. She and her team research and deploy the technologies that students, faculty, and staff use to learn, teach, and collaborate. On her blog, Martha writes about the tools and techniques she and her team are developing, she assesses how her local academic community is making use of those tools and techniques, and thinks broadly about the future of education.

Thomas Mahon is a Savile Row tailor. His shop in London caters to people who can spend two thousand pounds on a classic handmade suit. I’ll never be in the market for one of those, but if I were I’d be fascinated by Mahon’s blog, EnglishCut.com, which tells you everything you might want to know about Savile Row past and present, about how Mahan practices the craft of bespoke tailoring, and about how to buy and care for the garments he makes.

For Todd and Martha and Thomas, the benefits of claiming their Net identities in these ways run wide and deep. Over time, their online narratives become autobiographies read by friends, colleagues, or clients, and just as importantly, read by people who may one day become friends, colleagues, or clients.

In most cases, of course, the words, pictures, audio, and video you might choose to publish online won’t attract many readers, listeners, or viewers. That’s OK. The point is that the people they do attract will be exactly the right people: those who share your interests and goals.

We’ve always used the term ‘social networking’ to refer to the process of finding and connecting with those people. And that process has always depended on a fabric of trust woven most easily in the context of local communities and face-to-face interaction.

But our interests and goals aren’t merely local. We face global challenges that compel us to collaborate on a global scale. Luckily, the new modes of social networking can reach across the Internet to include people anywhere and everywhere. But if we’re going to trust people across the Internet, we’ll need to be able check their references. Self-published narrative is one crucial form of evidence. The public reaction to such narratives, readily discoverable thanks to search engines and citation indexes, is another.

Is this a new and strange new activity? From one perspective it is, and that’s why I can’t yet point to many other folks who’ve figured out appropriate and effective ways to be online, as Todd and Martha and Thomas have.

But from another perspective, Internet self-publishing is just a new way to do what we’ve been doing for tens of thousand years: telling stories to explain ourselves to one another, and to make sense of our world.


Simon Willison

Serving map tiles from SQLite with MBTiles and datasette-tiles

Working on datasette-leaflet last week re-kindled my interest in using Datasette as a GIS (Geographic Information System) platform. SQLite already has strong GIS functionality in the form of SpatiaLite and datasette-cluster-map is currently the most downloaded plugin. Most importantly, maps are fun! MBTiles I was talking to Tom MacWright on Monday and I mentioned that I'd been thinking about h

Working on datasette-leaflet last week re-kindled my interest in using Datasette as a GIS (Geographic Information System) platform. SQLite already has strong GIS functionality in the form of SpatiaLite and datasette-cluster-map is currently the most downloaded plugin. Most importantly, maps are fun!

MBTiles

I was talking to Tom MacWright on Monday and I mentioned that I'd been thinking about how SQLite might make a good mechanism for distributing tile images for use with libraries like Leaflet. "I might be able to save you some time there" he said... and he showed me MBTiles, a specification he started developing ten years ago at Mapbox which does exactly that - bundles tile images up in SQLite databases.

(My best guess is I read about MBTiles a while ago, then managed to forget about the spec entirely while the idea of using SQLite for tile distribution wedged itself in my head somewhere.)

The new datasette-tiles plugin

I found some example MBTiles files on the internet and started playing around with them. My first prototype used the datasette-media plugin, described here previously in Fun with binary data and SQLite. I used some convoluted SQL to teach it that hits to /-/media/tiles/{z},{x},{y} should serve up content from the tiles table in my MBTiles database - you can see details of that prototype in this TIL: Serving MBTiles with datasette-media.

The obvious next step was to write a dedicated plugin: datasette-tiles. Install it and run Datasette against any MBTiles database file and the plugin will set up a /-/tiles/db-name/z/x/y.png endpoint that serves the specified tiles.

It also adds a tile explorer view with a pre-configured Leaflet map. Here's a live demo serving up a subset of Stamen's toner map - just zoom levels 6 and 7 for the country of Japan.

Here's how to run this on your own computer:

# Install Datasette brew install datasette # Install the plugin datasette install datasette-tiles # Download the japan-toner.db database curl -O https://datasette-tiles-demo.datasette.io/japan-toner.db # Launch Datasette and open a browser datasette japan-toner.db -o # Use the cog menu to access the tile explorer # Or visit http://127.0.0.1:8001/-/tiles/japan-toner Creating MBTiles files with my download-tiles tool

A sticking point when I started playing with MBTiles was finding example files to work with.

After some digging, I came across the amazing HOT Export Tool. It's a project by the Humanitarian OpenStreetMap Team that allows anyone to export subsets of data from OpenStreetMap in a wide variety of formats, including MBTiles.

I filed a minor bug report against it, and in doing so took a look at the source code (it's all open source)... and found the code that assembles MBTiles files. It uses another open source library called Landez, which provides functions for downloading tiles from existing providers and bundling those up as an MBTiles SQLite file.

I prefer command-line tools for this kind of thing over using Python libraries directly, so I fired up my click-app cookiecutter template and built a thin command-line interface over the top of the library.

The new tool is called download-tiles and it does exactly that: downloads tiles from a tile server and creates an MBTiles SQLite database on disk containing those tiles.

Please use this tool responsibly. Downloading large numbers of tiles is bad manners. Be sure to familiarize yourself with the OpenStreetMap Tile Usage Policy, and use the tool politely when pointing it at other tile servers.

Basic usage is as follows:

download-tiles world.mbtiles

By default the tool pulls tiles from OpenStreetMap. The above command will fetch zoom levels 0-3 of the entire world - 85 tiles total, well within acceptable usage limits.

Various options (described in the README) can be used to customize the tiles that are downloaded. Here's how I created the japan-toner.db demo database, linked to above:

download-tiles japan-toner.mbtiles \ --zoom-levels 6-7 \ --country Japan \ --tiles-url "http://{s}.tile.stamen.com/toner/{z}/{x}/{y}.png" \ --tiles-subdomains "a,b,c,d" \ --attribution 'Map tiles by Stamen Design, under CC BY 3.0. Data by OpenStreetMap, under CC BY SA.'

The --country Japan option here looks up the bounding box for Japan using Nominatim. --zoom-levels 6-7 fetches zoom levels 6 and 7 (in this case that makes for 193 tiles total). --tiles-url and --tiles-subdomain configure the tile server to fetch them from. The --attribution option bakes that string into the metadata table for the database - which is then used to display it correctly in the tile explorer (and eventually in other Datasette plugins).

datasette-basemap

Out of the box, Datasette's current Leaflet plugins (datasette-cluster-map, datasette-leaflet-geojson and so on) serve tiles directly from the OpenStreetMap tile server.

I've never felt particularly comfortable about this. Users can configure the plugins to run against other tile servers, but pointing to OpenStreetMap as a default was the easiest way to ensure these plugins would work for people who just wanted to try them out.

Now that I have the tooling for bundling map subsets, maybe I can do better.

datasette-basemap offers an alternative: it's a plugin that bundles a 22.7MB SQLite file containing zoom levels 0-6 of OpenStreetMap - 5,461 tiles total.

Running pip install datasette-basemap (or datasette install datasette-basemap) will install the plugin, complete with that database - and register it with Datasette.

Start Datasette with the plugin installed and /basemap will expose the bundled database. Install datasette-tiles and you'll be able to browse it as a tile server: here's a demo.

(I recommend also installing datasette-render-images so you can see the tile images themselves in the regular table view, like this.)

Zoom level 6 is close enough that major cities and the roads between them are visible, for all of the countries in the world. Not bad for 22.7MB!

This is the first time I've built a Datasette plugin that bundles a full SQLite database as part of the Python package. The pattern seems to work well - I'm excited to explore it further with other projects.

Bonus feature: tile stacks

I added one last feature to datasette-tiles before writing everything up for my blog. I'm calling this feature tile stacks - it lets you serve tiles from multiple MBTiles files, falling back to other files if a tile is missing.

Imagine you had a low-zoom-level world map (similar to datasette-basemap) and a number of other databases providing packages of tiles for specific countries or cities. You could run Datasette like this:

datasette basemap.mbtiles japan.mbtiles london.mbtiles tokyo.mbtiles

Hitting /-/tiles-stack/1/1/1.png would seek out the specified tile in the tokyo.mbtiles file, then fall back to london.mbtiles and then japan.mbtiles and finally basemap.mbtiles if it couldn't find it.

For a demo, visit https://datasette-tiles-demo.datasette.io/-/tiles-stack and zoom in on Japan. It should start to display the Stamen toner map once you get to zoom levels 6 and 7.

Next steps

I've been having a lot of fun exploring MBTiles - it's such a natural fit for Datasette, and it's exciting to be able to build new things on top of nearly a decade of innovation by other geo-hackers.

There are plenty of features missing from datasette-tiles.

It currently only handles .png image data, but the MBTiles 1.3 specification also defines .jpg and .webp tiles, plus vector tiles using Mapbox's .pbf gzip-compressed protocol buffers.

UTFGrid is a related specification for including "rasterized interaction data" in MBTiles databases - it helps efficiently provide maps with millions of embedded objects.

As a newcomer to the MBTiles world I'd love to hear suggestions for new features and feedback on how I can improve what I've got so far in the datasette-tiles issues.

Being able to serve your own map tiles like this feels very much in the spirit of the OpenStreetMap project. I'm looking forward to using my own tile subsets for any future projects that fit within a sensible tile subset.

Wednesday, 03. February 2021

Information Answers

Applying for, and being, a MyData Operator

I’m on a panel this afternoon at this Canadian Data Privacy Week event; the subject I’m due to discuss is as per the title above – […]
I’m on a panel this afternoon at this Canadian Data Privacy Week event; the subject I’m due to discuss is as per the title above – […]

MyDigitalFootprint

What is the purpose of data? V2

We continually move towards better data-led decisions; however, we can easily ask our dataset’s wrong question. Without understanding “What is the purpose of data” on which we are basing decisions and judgements, it is easy to get an answer that is not in the data. How can we understand if our direction, Northstar or decision is a good one?  Why am I interested in this? I am focusing on how
We continually move towards better data-led decisions; however, we can easily ask our dataset’s wrong question. Without understanding “What is the purpose of data” on which we are basing decisions and judgements, it is easy to get an answer that is not in the data. How can we understand if our direction, Northstar or decision is a good one?  Why am I interested in this? I am focusing on how we improve governance and oversight in a data-led world. 

I wrote a lengthy article on Data is Data. It was a kickback at the analogies that data is oil, gold, labour, sunlight - data is not. Data is unique; it has unique characteristics. That article concluded that the word “Data” is also part of the problem, but we should think of data as if discovering a new element with unique characteristics.  

Data is a word, and it is part of the problem.  Data doesn’t have meaning or shape, and data will not have meaning unless we can give it context. As Theodora Lau eloquently put it; if her kiddo gets 10 points in a test today (data as a state), the number 10 has no meaning, unless we say, she scored 10 points out of 10 in the test today (data is information). And even then, we still need to explain the type of test (data is knowledge) and what to do next or how to improve (data is insights).  Each of these is a “data” point, and we don’t differentiate the use of the word “data” in these contexts.

Data’s most fundamental representation is “state” where it represents the particular condition something is in at a specific time.  I love Hugh’s work @gapingvoid (below) representation  Information is knowing that there are different “states” (on/off). Knowledge is finding patterns and connections.  Insight knows comparatives to state. Wisdom is the journey.  We live in the hope that the data we have will have an impact.

For a while, the data community has rested on two key characteristics of data: non-rivalrous (which plays havoc with our current understanding of ownership) and non-fungible (which is true if you assume that data carries information.)  Whilst these are both accurate observations; they are not that good as universal characteristics.

Non-rivalrous. Economists call an item that can only be used by one person at a time as "rivalrous." Money and capital are rivalrous. Data is non-rivalrous as a single item of data can simultaneously fuel multiple algorithms, analytics, and applications.  This is, however, not strictly true. Numerous perfect copies of “data” can be used simultaneously because the marginal cost of reproduction and sharing is zero.   

Non-fungible. When you can substitute one item for another, they are said to be fungible.  One sovereign bill can be replaced for another sovereign bill of the same value; one barrel of oil is the same as the next.  So the thinking goes, data is non-fungible and cannot be substituted because it carries information.  However, if your view is that data carries state (the particular condition that something is in at a specific time), then data is fungible. Higher-level ideals of data that is processed (information, knowledge, insights) are increasingly non-fungible.

Money as a framework to explore the purpose of data  

Sovereign currency (FIAT), money in this setting, has two essential characteristics.  It has rivalrous and fungible.  Without these foundational characteristics, money cannot fulfil its original purpose (it has many others now); a trusted exchange medium.  Money removes the former necessity of a direct barter, where equal value had to be established, and the two or more parties had to meet for an exchange.  What is interesting is that there are alternatives to FIAT which exploit other properties.  Because of fraud, we have to have security features, and there is a race to build the most secure wall.


[Just as a side note - money is an abstraction and part of the rationale for a balance sheet was to try to connect the abstraction back to real things. Not sure that works any more]

Revising the matrix “what problem is to be solved?” 

Adding these other options of exchange onto the matrix, we have a different way to frame what problem each type of currency offers as a method of exchange mechanism. This is presented in the chart below.  Sand and beans can be used, but they provide a messy tool compared to a sovereign currency.  Crypto works, and it solves the problem, but without exchange to other currencies, it had fundamental limits.  

If we now add digital data and other aspects of our world onto the matrix, we have a different perspective. We all share gravity, sunsets and broadcast TV/ radio on electromagnetic waves.  However, only one atom can be used at a time, and that atom is not-interchangeable (to get the same outcome.)  The point is that digital data is not in the same quadrant as sovereign currency and electrons as a beautiful solution based on being fungible and rivalrous.  

In the broadest definition of data which is “state”; chemical, atoms, gravity, electrons have state and therefore are also data.  To be clear will now use Digital Data to define our focus and not all data. 

These updates to the matrix highlight that, if data is non-rivalrous and non-fungible, these characteristics mean that is is very unclear to what problem digital data is solving.  We see this all the time in the digital data market, as we cannot agree on what “data” is, it is messy. 

The question for us as a digital data community is; “what are the axis [characteristics] that mean digital data is in the top corner of a matrix? This is where digital data is a beautiful solution to a defined problem, given that digital data is at its core is “knowing state.”  I explored this question on a call with Scott David, and we ended up resting on “Rights and Attestation” as the two axes 

Rights in this context are that you have gained rights from the Parties.  What and how those rights were acquired is not the question; it is just that you have the rights you need to do what you need to do.

Attestation in this context is the evidence or proof of something.  It is that you know what you have is true and that you can prove the state exists. How you do this is not the point; it is just you know it is provable.

As we saw with the money example, data will never have these (rights and attestation) characteristics exclusively; it is just when it has them, data is most purposeful.  Without attestation, the data you have is compromised, and any conclusions you reach may not be true or real. Continually we have to test both our assumptions and the provability of our digital data.   Rights are different as rights are not correlated with data quality, but rights may help resolve ownership issues.  A business built without rights to the data they are using is not stable or sustainable.  How and if those Rights were obtained ethically are matters to be investigated.  Interestingly, these characteristics (rights and attestation)  would readily fit into existing risk and audit frameworks. 

I have a specific focus on ESG, sustainability, data for decision making, and better data for sharing.  Given that most comparative ESG data is from public reports (creative commons or free of rights), it is essential to note there is a break in the attestation.  ESG data right now is in the least useful data bucket for decision making, but we are making critical investment decisions on this analysis data set. It is something that we have to address. 


In summary

If the purpose of data is “to share state” then the two essential characteristics data must have are rights and attestation.   Further, as data becomes information (knowing state), knowledge (patterns of states), insight (context in states) and wisdom - these characteristics of rights and attestation matter even more.  If you are making decisions on data that you don’t know if it is true or have the rights to it, becomes a dangerous place. 

As a side, there is lots of technology and processes to know if the state is true (as in correct - not truth); if the state sensing is working and the level of accuracy; if the state at both ends has the same representation (providence/ lineage ); if it is secure; if we can gain information; if we can combine data sets and what the ontology is.  But these are not fundamental characteristics; they are supportive and ensure we have a vibrant ecosystem of digital data.   

I am sure there are other labels for such a matrix and interested in your views, thoughts and comments. 

Simon Willison

Cleaning Up Your Postgres Database

Cleaning Up Your Postgres Database Craig Kerstiens provides some invaluable tips on running an initial check of the health of a PostgreSQL database, by using queries against the pg_statio_user_indexes table to find the memory cache hit ratio and the pg_stat_user_tables table to see what percentage of queries to your tables are using an index. Via @craigkerstiens

Cleaning Up Your Postgres Database

Craig Kerstiens provides some invaluable tips on running an initial check of the health of a PostgreSQL database, by using queries against the pg_statio_user_indexes table to find the memory cache hit ratio and the pg_stat_user_tables table to see what percentage of queries to your tables are using an index.

Via @craigkerstiens

Monday, 01. February 2021

MyDigitalFootprint

Does data have a purpose?

We are continually moving towards better data-led decisions, however, without understanding “What is the purpose of data / Does data have a purpose”   on which we are basing decisions and judgements, it is hard to understand if our north star (a good decision) is a good one.  Why am I interested in this, as I am focusing on how we do governance and oversight better in a data-led world.&
We are continually moving towards better data-led decisions, however, without understanding “What is the purpose of data / Does data have a purpose”   on which we are basing decisions and judgements, it is hard to understand if our north star (a good decision) is a good one.  Why am I interested in this, as I am focusing on how we do governance and oversight better in a data-led world. 

I wrote a lengthy article on Data is Data. It was a kickback at the analogies that data is oil, gold, labour, sunlight - data is not. Data is unique; it has unique characteristics. That article concluded that the word “Data” is also part of the problem, but we should think of data as if discovering a new element with unique characteristics.   

For a while, the data community has rested on two key characteristics of data: non-rivalrous (which plays havoc with our current understanding of ownership) and non-fungible (which is true if you assume that data carries information.)  Whilst these are both accurate observations; they are not universal.

Non-rivalrous. Economists call an item that can only be used by one person at a time. "rivalrous." Money and capital are rivalrous. Data is non-rivalrous as a single item of data can simultaneously fuel multiple algorithms, analytics, and applications.  This is, however, not strictly true. It is that numerous perfect copies of data can be used simultaneously.   

Non-fungible. When you can substitute one item for another, they are said to be fungible.  One sovereign bill can be replaced for another sovereign bill of the same value; one barrel of oil is the same as the next.  So the thinking goes, data is non-fungible and cannot be substituted because it carries information.  However, if your view is that data carries state (the particular condition that something is in at a specific time), then data is fungible. 

I love Hugh’s work, and @gapingvoid nailed this.  Data’s most basic representation is “state” where is represents the particular condition something is in at a specific time.  Information is knowing that there are different “states” (on/off). Knowledge is finding patterns and connections.  Insights know there is an exception to the current state. Wisdom is the journey.  The point is that non-rivalrous and non-fungible is not good enough as “data” is the mechanism for representation of all these properties in a digital world.  


Money as a framework to explore the purpose of data  

Sovereign FIAT currency, money in this setting, has two essential characteristics.  It has rivalrous and fungible.  Without these foundational characteristics, money cannot fulfil its purpose; a trusted medium of exchange.  Money removes the former necessity of a direct barter, where equal value had to be established, and the two or more parties had to meet.  What is interesting is that there are alternatives to FIAT which exploit other properties.  Because of fraud, we have to have security features, and there is a race to build the most secure wall.


[Just as a side note - money is an abstraction and part of the rationale for a balance sheet was to try to connect the abstraction back to real things. Not sure that works any more]

Revising the matrix but thinking about what problem is to be solved. 

We are now adding data and other ideals on the matrix, as a different way to frame data. 

These updates to the matrix highlight that, if data is non-rivalrous and non-fungible, these characteristics mean that is is very unclear to what problem data is solving.  Indeed we see this all the time in the data market, as we cannot agree on what data is, it is messy. 

The question for us as a data community is; “what are the axis [characteristics] that mean data is in the top corner of a matrix? This is where data is a beautiful solution to a defined problem, given that data is at its core is “share state.”  We explored this question and proposed Rights and Attestation as the two axes on a call with Scott David.  

Rights in this context are that you have gained rights from the Parties.  What and how those rights were acquired is not the question; it is just that you have the rights you need to do what you need to do.

Attestation in this context is the evidence or proof of something.  It is that you know what you have is true and that you can prove the state exists.

As we saw with the money example, data will never have these characteristics exclusively; it is just when it has them, data is most purposeful.  Without attestation, the data you have is compromised, and any conclusions you reach may not be true or real. Continually we have to test both our assumptions and the provability of the data.   Rights are different as rights are not correlated with data quality.  A business built without rights to the data they are using is not stable or sustainable.  How and if those Rights were obtained ethically are issues to be investigated.  Interestingly, these characteristics would readily fit into a risk and audit framework today. 

I have a specific focus on ESG, sustainability and better data for decision making, and better data for sharing.  Given that most comparative ESG data is from public reports (creative commons or free of rights), but more importantly, there is a break in the attestation.  ESG data right now is in the least useful data bucket for decision making, but we are making critical investment decisions on this analysis data set. It is something that we have to address. 


In summary

If the purpose of data is “to share state” then the two essential characteristics data must have are rights and attestation.   Further, as data becomes information (knowing state), knowledge (patterns of states), insight (issues in states) and wisdom - these characteristics of rights and attestation matter even more.  If you are making decisions on data that you don’t know if it is true or have the rights to it, becomes a dangerous place. 

As a side, there is lots of technology and processes to know if the state is true (as in correct - not truth); if the state sensing is working and the level of accuracy; if the state at both ends has the same representation (providence/ lineage ); if it is secure; if we can gain information; if we can combine data sets and what the ontology is.  But these are not so fundamental; they are supportive that make the ecosystem of data work.   

We are sure there are other labels for such a matrix and interested in your views, thoughts and comments. 




Simon Willison

JMeter Result Analysis using Datasette

JMeter Result Analysis using Datasette NaveenKumar Namachivayam wrote a detailed tutorial on using Datasette (on Windows) and csvs-to-sqlite to analyze the results of JMeter performance test runs and then publish them online using Vercel. Via @QAInsights

JMeter Result Analysis using Datasette

NaveenKumar Namachivayam wrote a detailed tutorial on using Datasette (on Windows) and csvs-to-sqlite to analyze the results of JMeter performance test runs and then publish them online using Vercel.

Via @QAInsights

Sunday, 31. January 2021

Simon Willison

Quoting Tim Cook

Technology does not need vast troves of personal data stitched together across dozens of websites and apps in order to succeed. Advertising existed and thrived for decades without it, and we're here today because the path of least resistance is rarely the path of wisdom. — Tim Cook

Technology does not need vast troves of personal data stitched together across dozens of websites and apps in order to succeed. Advertising existed and thrived for decades without it, and we're here today because the path of least resistance is rarely the path of wisdom.

Tim Cook


Weeknotes: datasette-leaflet, datasette-plugin cookiecutter upgrades

This week I shipped Datasette 0.54, sent out the latest Datasette Newsletter and then mostly worked on follow-up projects. datasette-leaflet Datasette 0.54 introduced support for JavaScript modules. datasette-leaflet aims to solve a growing problem: there are now three Datasette plugins that use the Leaflet JavaScript mapping library, and all three were loading it in different ways and using d

This week I shipped Datasette 0.54, sent out the latest Datasette Newsletter and then mostly worked on follow-up projects.

datasette-leaflet

Datasette 0.54 introduced support for JavaScript modules. datasette-leaflet aims to solve a growing problem: there are now three Datasette plugins that use the Leaflet JavaScript mapping library, and all three were loading it in different ways and using different versions.

Those three plugins - datasette-leaflet-freedraw, datasette-leaflet-geojson and datasette-cluster-map - now all depend on datasette-leaflet. This should mean that even if multiple plugins are installed they will all load and use the same copy of the Leaflet library.

There's just one problem: datasette-leaflet-freedraw uses the Leaflet.FreeDraw module, which it turns out already bundles its own copy of Leaflet. I still need to figure out how to build a version of that package which can take advantage of the shared copy of Leaflet - see open issue #5.

datasette-plugin and cookiecutter tests

One of the tricker aspects of building a Datasette plugin is bundling static assets such as JavaScript and CSS in the plugin to be distributed via PyPI. I know this is hard because I frequently mess up the package_data= section of setup.py and have to ship another release to fix my mistake!

I've upgraded the datasette-plugin cookiecutter template to help with this. The template now asks you if you want to create static/ or template/ directories as part of the interactive configuration - if you say yes to either of those it will create the directories for you and configure package_data= to correctly bundle any files contained within them.

This lead to a couple of new TILs: Testing cookiecutter templates with pytest and Conditionally creating directories in cookiecutter.

datasette-export-notebook improvements

datasette-export-notebook is a plugin that adds copy-and-paste instructions for exporting data from Datasette to Jupyter or Observable notebooks.

The "stream all rows" option for Jupyter uses Datasette's CSV streaming export, which has the downside of losing any type information.

As of issue #8 the copy-and-paste code for Jupyter now uses the dtype= option to specify the correct types.

You can see it in action on this page, where the Jupyter example code now looks like this:

df = pandas.read_csv( "https://covid-19.datasettes.com/covid/us_census_county_populations_2019.csv?_stream=on", dtype={ "fips": int, "population": int, } ) Releases this week datasette-export-notebook: 0.3 - 2021-01-29
Datasette plugin providing instructions for exporting data to Jupyter or Observable datasette-debug-asgi: 1.0 - 2021-01-29
Datasette plugin for dumping out the ASGI scope datasette-template-sql: 1.0.2 - 2021-01-29
Datasette plugin for executing SQL queries from templates datasette-cluster-map: 0.17 - 2021-01-29
Datasette plugin that shows a map for any data with latitude/longitude columns datasette-leaflet-geojson: 0.8 - 2021-01-26
Datasette plugin that replaces any GeoJSON column values with a Leaflet map. datasette-leaflet-freedraw: 0.2.1 - 2021-01-25
Draw polygons on maps in Datasette datasette-leaflet: 0.2 - 2021-01-25
Datasette plugin adding the Leaflet JavaScript library datasette: 0.54 - 2021-01-25
An open source multi-tool for exploring and publishing data TIL this week Testing cookiecutter templates with pytest Conditionally creating directories in cookiecutter

Saturday, 30. January 2021

Identity Woman

Internet of People is doing false advertising

I just learned about the internet of people project. It seems cool…I need to dig in a bit more…but already there is a huge red flag/disconnect for me. These are the guys who are signing off on this post they put a picture of themselves on zoom. These are the women (many of them of […] The post Internet of People is doing false advertising appeared first on Identity Woman.

I just learned about the internet of people project. It seems cool…I need to dig in a bit more…but already there is a huge red flag/disconnect for me. These are the guys who are signing off on this post they put a picture of themselves on zoom. These are the women (many of them of […]

The post Internet of People is doing false advertising appeared first on Identity Woman.


Mike Jones: self-issued

Be part of the Spring 2021 IIW!

Are you registered for the Internet Identity Workshop (IIW) yet? As I wrote a decade, a year, and a day ago, “It’s where Internet identity work gets done.” That remains as true now is it was then! As a personal testimonial, I wrote this to the IIW organizers after the 2020 IIWs: “Thanks again for […]

Are you registered for the Internet Identity Workshop (IIW) yet? As I wrote a decade, a year, and a day ago, “It’s where Internet identity work gets done.” That remains as true now is it was then!

As a personal testimonial, I wrote this to the IIW organizers after the 2020 IIWs:

“Thanks again for running the most engaging and successful virtual meetings of the year (by far!). While I’ve come to dread most of the large virtual meetings, IIW online remains true to the spirit of the last 15 years of useful workshops. Yes, I miss talking to Rich and the attendees in the coffee line and having impromptu discussions throughout, and we’ll get back to that in time, but the sessions remain useful and engaging.”

I’m also proud that Microsoft is continuing its 15-year tradition of sponsoring the workshop. Rather than buying dinner for the attendees (the conversations at the dinners were always fun!), we’re sponsoring scholarships for those that might otherwise not be able to attend, fostering an even more interesting and diverse set of viewpoints at the workshop.

I hope to see you there!

Friday, 29. January 2021

Simon Willison

Making GitHub’s new homepage fast and performant

Making GitHub’s new homepage fast and performant A couple of really clever tricks in this article by Tobias Ahlin. The first is using IntersectionObserver in conjunction with the video preload="none" attribute to lazily load a video when it scrolls into view. The second is an ingenious trick to create an efficiently encoded transparent JPEG image: embed the image in a SVG file twice, once as the

Making GitHub’s new homepage fast and performant

A couple of really clever tricks in this article by Tobias Ahlin. The first is using IntersectionObserver in conjunction with the video preload="none" attribute to lazily load a video when it scrolls into view. The second is an ingenious trick to create an efficiently encoded transparent JPEG image: embed the image in a SVG file twice, once as the image and once as a transparency mask.

Thursday, 28. January 2021

Information Answers

BLTS > TBSL, the order matters

OK, yes the post heading is a bit obscure and for a specific audience; so let me explain. Over in the MyData.org community (and other such […]
OK, yes the post heading is a bit obscure and for a specific audience; so let me explain. Over in the MyData.org community (and other such […]

Werdmüller on Medium

Your 401(k) hates you

How a retirement vehicle from the seventies is crippling America Continue reading on Medium »

How a retirement vehicle from the seventies is crippling America

Continue reading on Medium »


8 simple ways to get the most out of today

It’s a brand new day! Time to seize it. Continue reading on Medium »

It’s a brand new day! Time to seize it.

Continue reading on Medium »

Tuesday, 26. January 2021

Doc Searls Weblog

Just in case you feel safe with Twitter

Just got a press release by email from David Rosen (@firstpersonpol) of the Public Citizen press office. The headline says “Historic Grindr Fine Shows Need for FTC Enforcement Action.” The same release is also a post in the news section of the Public Citizen website. This is it: WASHINGTON, D.C. – The Norwegian Data Protection Agency today fined Grindr $11.7 million&nb

Just got a press release by email from David Rosen (@firstpersonpol) of the Public Citizen press office. The headline says “Historic Grindr Fine Shows Need for FTC Enforcement Action.” The same release is also a post in the news section of the Public Citizen website. This is it:

WASHINGTON, D.C. – The Norwegian Data Protection Agency today fined Grindr $11.7 million following a Jan. 2020 report that the dating app systematically violates users’ privacy. Public Citizen asked the Federal Trade Commission (FTC) and state attorneys general to investigate Grindr and other popular dating apps, but the agency has yet to take action. Burcu Kilic, digital rights program director for Public Citizen, released the following statement:

“Fining Grindr for systematic privacy violations is a historic decision under Europe’s GDPR (General Data Protection Regulation), and a strong signal to the AdTech ecosystem that business-as-usual is over. The question now is when the FTC will take similar action and bring U.S. regulatory enforcement in line with those in the rest of the world.

“Every day, millions of Americans share their most intimate personal details on apps like Grindr, upload personal photos, and reveal their sexual and religious identities. But these apps and online services spy on people, collect vast amounts of personal data and share it with third parties without people’s knowledge. We need to regulate them now, before it’s too late.”

The first link goes to Grindr is fined $11.7 million under European privacy law, by Natasha Singer (@NatashaNYT) and Aaron Krolik. (This @AaronKrolik? If so, hi. If not, sorry. This is a blog. I can edit it.) The second link goes to a Public Citizen post titled Popular Dating, Health Apps Violate Privacy.

In the emailed press release, the text is the same, but the links are not. The first is this:

https://default.salsalabs.org/T72ca980d-0c9b-45da-88fb-d8c1cf8716ac/25218e76-a235-4500-bc2b-d0f337c722d4

The second is this:

https://default.salsalabs.org/Tc66c3800-58c1-4083-bdd1-8e730c1c4221/25218e76-a235-4500-bc2b-d0f337c722d4

Why are they not simple and direct URLs? And who is salsalabs.org?

You won’t find anything at that link, or by running a whois on it. But I do see there is a salsalabs.com, which has  “SmartEngagement Technology” that “combines CRM and nonprofit engagement software with embedded best practices, machine learning, and world-class education and support.” since Public Citizen is a nonprofit, I suppose it’s getting some “smart engagement” of some kind with these links. PrivacyBadger tells me Salsalabs.com has 14 potential trackers, including static.ads.twitter.com.

My point here is that we, as clickers on those links, have at best a suspicion about what’s going on: perhaps that the link is being used to tell Public Citizen that we’ve clicked on the link… and likely also to help target us with messages of some sort. But we really don’t know.

And, speaking of not knowing, Natasha and Aaron’s New York Times story begins with this:

The Norwegian Data Protection Authority said on Monday that it would fine Grindr, the world’s most popular gay dating app, 100 million Norwegian kroner, or about $11.7 million, for illegally disclosing private details about its users to advertising companies.

The agency said the app had transmitted users’ precise locations, user-tracking codes and the app’s name to at least five advertising companies, essentially tagging individuals as L.G.B.T.Q. without obtaining their explicit consent, in violation of European data protection law. Grindr shared users’ private details with, among other companies, MoPub, Twitter’s mobile advertising platform, which may in turn share data with more than 100 partners, according to the agency’s ruling.

Before this, I had never heard of MoPub. In fact, I had always assumed that Twitter’s privacy policy either limited or forbid the company from leaking out personal information to advertisers or other entities. Here’s how its Private Information Policy Overview begins:

You may not publish or post other people’s private information without their express authorization and permission. We also prohibit threatening to expose private information or incentivizing others to do so.

Sharing someone’s private information online without their permission, sometimes called doxxing, is a breach of their privacy and of the Twitter Rules. Sharing private information can pose serious safety and security risks for those affected and can lead to physical, emotional, and financial hardship.

On the MoPub site, however, it says this:

MoPub, a Twitter company, provides monetization solutions for mobile app publishers and developers around the globe.

Our flexible network mediation solution, leading mobile programmatic exchange, and years of expertise in mobile app advertising mean publishers trust us to help them maximize their ad revenue and control their user experience.

The Norwegian DPA apparently finds a conflict between the former and the latter—or at least in the way the latter was used by Grinder (since they didn’t fine Twitter).

To be fair, Grindr and Twitter may not agree with the Norwegian DPA. Regardless of their opinion, however, by this point in history we should have no faith that any company will protect our privacy online. Violating personal privacy is just too easy to do, to rationalize, and to make money at.

To start truly facing this problem, we need start with a simple fact: If your privacy is in the hands of others alone, you don’t have any. Getting promises from others not to stare at your naked self isn’t the same as clothing. Getting promises not to walk into your house or look in your windows is not the same as having locks and curtains.

In the absence of personal clothing and shelter online, or working ways to signal intentions about one’s privacy, the hands of others alone is all we’ve got. And it doesn’t work. Nor do privacy laws, especially when enforcement is still so rare and scattered.

Really, to potential violators like Grindr and Twitter/MoPub, enforcement actions like this one by the Norwegian DPA are at most a little discouraging. The effect on our experience of exposure is still nil. We are exposed everywhere, all the time, and we know it. At best we just hope nothing bad happens.

The only way to fix this problem is with the digital equivalent of clothing, locks, curtains, ways to signal what’s okay and what’s not—and to get firm agreements from others about how our privacy will be respected.

At Customer Commons, we’re starting with signaling, specifically with first party terms that you and I can proffer and sites and services can accept.

The first is called P2B1, aka #NoStalking. It says “Just give me ads not based on tracking me.” It’s a term any browser (or other tool) can proffer and any site or service can accept—and any privacy-respecting website or service should welcome.

Making this kind of agreement work is also being addressed by IEEE7012, a working group on machine-readable personal privacy terms.

Now we’re looking for sites and services willing to accept those terms. How about it, Twitter, New York Times, Grindr and Public Citizen? Or anybody.

DM us at @CustomerCommons and we’ll get going on it.

 


Phil Windley's Technometria

Generative Identity

Summary: Generative identity allows us to live digital lives with dignity and effectiveness, contemplates and addresses the problems of social inclusion, and supports economic equality to everyone around the globe. This article describes the implementation self-sovereign identity through protocol-mediated credential exchange on the self-sovereign internet, examines its properties, and argues

Summary: Generative identity allows us to live digital lives with dignity and effectiveness, contemplates and addresses the problems of social inclusion, and supports economic equality to everyone around the globe. This article describes the implementation self-sovereign identity through protocol-mediated credential exchange on the self-sovereign internet, examines its properties, and argues for it generative nature from those properties.

The Generative Self-Sovereign Internet explored the generative properties of the self-sovereign internet, a secure overlay network created by DID connections. The generative nature of the self-sovereign internet is underpinned by the same kind of properties that make the internet what it is, promising a more secure and private, albeit no less useful, internet for tomorrow.

In this article, I explore the generativity of self-sovereign identity—specifically the exchange of verifiable credentials. One of the key features of the self-sovereign internet is that it is protocological—the messaging layer supports the implementation of protocol-mediated interchanges on top of it. This extensibility underpins its generativity. Two of the most important protocols defined on top of the self-sovereign internet support the exchange of verifiable credentials as we'll see below. Together, these protocols work on top of the the self-sovereign internet to give rise to self-sovereign identity through a global identity metasystem.

Verifiable Credentials

While the control of self-certifying identifiers in the form of DIDs is the basis for the autonomy of the self-sovereign internet, that autonomy is made effective through the exchange of verifiable credentials. Using verifiable credentials, an autonomous actor on the self-sovereign internet can prove attributes to others in a way they can trust. Figure 1 shows the SSI stack. The self-sovereign internet is labeled "Layer Two" in this figure. Credential exchange happens on top of that in Layer Three.

Figure 1: SSI Stack (click to enlarge)

Figure 2 shows how credentials are exchanged. In this diagram, Alice has DID-based relationships with Bob, Carol, Attestor.org and Certiphi.com. Alice has received a credential from Attestor.org. The credential contains attributes that Attestor.org is willing to attest belong to Alice. For example, Attestor might be her employer attesting that she is an employee. Attestor likely gave her a credential for their own purposes. Maybe Alice uses it for passwordless login at company web sites and services and to purchase meals at the company cafeteria. She might also use it at partner websites (like the benefits provider) to provide shared authentication without federation (and it's associated infrastructure). Attestor is acting as a credential issuer. We call Alice a credential holder in this ceremony. The company and partner websites are credential verifiers. Credential issuance is a protocol that operates on top of the self-sovereign internet.

Figure 2: Credential Exchange (click to enlarge)

Even though Attestor.org issued the credential to Alice for its own purposes, she holds it in her wallet and can use it at other places besides Attestor. For example, suppose she is applying for a loan and her bank, Certiphi, who wants proof that she's employed and has a certain salary. Alice could use the credential from Attestor to prove to Certiphi that she's employed and that her salary exceeds a given threshold1. Certiphi is also acting as a credential verifier. Credential proof and verification is also protocol that operates on top of the self-sovereign internet. As shown in Figure 2, individuals can also issue and verify credentials.

We say Alice "proved" attributes to Certiphi from her credentials because the verification protocol uses zero knowledge proof to support the minimal disclosure of data. Thus the credential that Alice holds from Attestor might contain a rich array of information, but Alice need only disclose the information that Certiphi needs for her loan. In addition, the proof process ensures that Alice can't be correlated though the DIDs she has shared with others. Attribute data isn't tied to DIDs or the keys that are currently assigned to the DID. Rather than attributes bound to identifiers and keys, Alice's identifiers and keys empower the attributes.

Certiphi can validate important properties of the credential. Certiphi is able to validate the fidelity of the credential by reading the credential definition from the ledger (Layer One in Figure 1), retrieving Attestor's public DID from the credential definition, and resolving it to get Attestor.org's public key to check the credential's signature. At the same time, the presentation protocol allows Certiphi to verify that the credential is being presented by the person it was issued to and that it hasn't been revoked (using a revocation registry store in Layer 1). Certiphi does not need to contact Attestor or have any prior business relationship to verify these properties.

The global identity metasystem, shown as the yellow box in Figure 1, comprises the ledger at Layer 1, the self-sovereign internet at Layer 2, and the credential exchange protocols that operate on top of it. Together, these provide the necessary features and characteristics to support self-sovereign identity.

Properties of Credential Exchange

Verifiable credentials have five important characteristics that mirror how credentials work in the offline world:

Credentials are decentralized and contextual. There is no central authority for all credentials. Every party can be an issuer, a holder, or a verifier. Verifiable credentials can be adapted to any country, any industry, any community, or any set of trust relationships. Credential issuers decide what data is contained in their credentials. Anyone can write credential schemas to the ledger. Anyone can create a credential definition based on any of these schemas. Verifiers make their own decisions about which credentials to accept—there's no central authority who determines what credentials are important or which are used for a given purpose. Verifiers do not need to contact issuers to perform verification—that's what the ledger is for. Credential verifiers don't need to have any technical, contractual, or commercial relationship with credential issuers in order to determine the credentials' fidelity. Credential holders are free to choose which credentials to carry and what information to disclose. People and organizations are in control of the credentials they hold and to determine what to share with whom.

These characteristics underlie several important properties that support the generativity of credential exchange. Here are the most important:

Private— Privacy by Design is baked deep into the architecture of the identity metasystem as reflected by several fundamental architectural choices:

Peer DIDs are pairwise unique and pseudonymous by default to prevent correlation. Personal data is never written to the ledgers at Layer 1 in Figure 1—not even in encrypted or hashed form. Instead, all private data is exchanged over peer-to-peer encrypted connections between off-ledger agents at Layer 2. The ledger is used for anchoring rather than publishing encrypted data. Credential exchange has built-in support for zero-knowledge proofs (ZKP) to avoid unnecessary disclosure of identity attributes. As we saw earlier, verifiers don’t need to contact the issuer to verify a credential. Consequently, the issuer doesn’t know when or where the credential is used.

Decentralized—decentralization follows directly from the fact that no one owns the infrastructure that supports credential exchange. This is the primary criterion for judging the degree of decentralization in a system. Rather, the infrastructure, like that of the internet, is operated by many organizations and people bound by protocol.

Heterarchical—a heterarchy is a "system of organization where the elements of the organization are unranked (non-hierarchical) or where they possess the potential to be ranked a number of different ways." Participants in credential exchange relate to each other as peers and are autonomous.

Interoperable—verifiable credentials have a standard format, readily accessible schemas, and a standard protocols for issuance, proving (presenting), and verification. Participants can interact with anyone else so long as they use tools that follow the standards and protocols. Credential exchange isn't a single, centralized system from a single vendor with limited pieces and parts. Rather, interoperability relies on interchangeable parts, built and operated by various parties. Interoperability supports substitutability, a key factor in autonomy and flexibility.

Substitutable—the tools for issuing, holding, proving, and verifying are available from multiple vendors and follow well-documented, open standards. Because these tools are interoperable, issuers, holders, and verifiers can choose software, hardware, and services without fear of being locked into a proprietary tool. Moreover, because many of the attributes the holder needs to prove (e.g. email address or even employer) will be available on multiple credentials, the holder can choose between credentials. Usable substitutes provide choice and freedom.

Flexible—closely related to substitutability, flexibility allows people to select appropriate service providers and features. No single system can anticipate all the scenarios that will be required for billions of individuals to live their own effective lives. The characteristics of credential exchange allow for context-specific scenarios.

Reliable and Censorship Resistant—people, businesses, and others must be able to exchange credentials without worrying that the infrastructure will go down, stop working, go up in price, or get taken over by someone who would do them harm. Substitutability of tools and credentials combined with autonomy makes the system resistant to censorship. There is no hidden third party or intermediary in Figure 2. Credentials are exchanged peer-to-peer.

Non-proprietary and Open—no one has the power to change how credentials are exchanged by fiat. Furthermore, the underlying infrastructure is less likely to go out of business and stop operation because its maintenance and operation are decentralized instead of being in the hands of a single organization. The identity metasystem has the same three virtues of the Internet that Doc Searls and Dave Weinberger enumerated as NEA: No one owns it, Everyone can use it, and Anyone can improve it. The protocols and code that enable the metasystem are open source and available for review and improvement.

Agentic—people can act as autonomous agents, under their self-sovereign authority. The most vital value proposition of self-sovereign identity is autonomy—not being inside someone else's administrative system where they make the rules in a one-sided way. Autonomy requires that participants interact as peers in the system, which the architecture of the metasystem supports.

Inclusive—inclusivity is more than being open and permissionless. Inclusivity requires design that ensures people are not left behind. For example, some people cannot act for themselves for legal (e.g. minors) or other (e.g. refugees) reasons. Support for digital guardianship ensures that those who cannot act for themselves can still participate.

Universal—successful protocols eat other protocols until only one survives. Credential exchange, built on the self-sovereign internet and based on protocol, has network effects that drive interoperability leading to universality. This doesn't mean that the metasystem will be mandated. Rather, one protocol will mediate all interaction because everyone in the ecosystem will conform to it out of self-interest.

The Generativity of Credential Exchange

Applying Zittrain's framework for evaluating generativity is instructive for understanding the generative properties of self-sovereign identity.

Capacity for Leverage

In Zittrain's words, leverage is the extent to which an object "enables valuable accomplishments that otherwise would be either impossible or not worth the effort to achieve." Leverage multiplies effort, reducing the time and cost necessary to innovate new capabilities and features.

Traditional identity systems have been anemic, supporting simple relationships focused on authentication and a few basic attributes their administrators need. They can't easily be leveraged by anyone but their owner. Federation through SAML or OpenID Connect has allowed the authentication functionality to be leveraged in a standard way, but authentication is just a small portion of the overall utility of a digital relationship.

One example of the capacity of credential exchange for leverage is to consider that it could be the foundation for a system that disintermediates platform companies like Uber, AirBnB, and the food delivery platforms. Platform companies build proprietary trust frameworks to intermediate exchanges between parties and charging exorbitant rents for what ought to be a natural interaction among peers. Credential exchange can open these trust frameworks up to create open marketplaces for services.

The next section on Adaptability lists a number of uses for credentials. The identity metasystem supports all these use cases with minimal development work on the part of issuers, verifiers, and holders. And because the underlying system is interoperable, an investment in the tools necessary to solve one identity problem with credentials can be leveraged by many others without new investment. The cost to define a credential is very low (often less than $100) and once the definition is in place, there is no cost to issue credentials against it. A small investment can allow an issuer to issue millions of credentials of different types for different use cases.

Adaptability

Adaptability can refer to a technology's ability to be used for multiple activities without change as well as its capacity for modification in service of new use cases. Adaptability is orthogonal to a technology's capacity for leverage. An airplane, for example, offers incredible leverage, allowing goods and people to be transported over long distances quickly. But airplanes are neither useful in activities outside transportation or easily modified for different uses. A technology that supports hundreds of use cases is more generative than one that is useful in only a few.

Identity systems based on credential exchange provide people with the means of operationalizing their online relationships by providing them the tools for acting online as peers and managing the relationships they enter into. Credential exchange allows for ad hoc interactions that were not or cannot be imagined a priori.

The flexibility of credentials ensures they can be used in a variety of situations. Every form or official piece of paper is a potential credential. Here are a few examples of common credentials:

Employee badges Drivers license Passport Wire authorizations Credit cards Business registration Business licenses College transcripts Professional licensing (government and private)

But even more important, every bundle of data transmitted in a workflow is a potential credential. Since credentials are just trustworthy containers for data, there are many more use cases that may not be typically thought of as credentials:

Invoices and receipts Purchase orders Airline or train ticket Boarding pass Certificate of authenticity (e.g. for art, other valuables) Gym (or any) membership card Movie (or any) tickets Insurance cards Insurance claims Titles (e.g. property, vehicle, etc.) Certificate of provenance (e.g. non-GMO, ethically sourced, etc.) Prescriptions Fractional ownership certificates for high value assets CO2 rights and carbon credit transfers Contracts

Since even a small business might issue receipts or invoices, have customers who use the company website, or use employee credentials, most businesses will define at least one credential and many will need many more. There are potentially tens of millions of different credential types. Many will use common schemas but each credential from a different issuer constitutes a different identity credential for a different context.

With the ongoing work in Hyperledger Aries, these use cases expand even further. With a “redeemable credentials” feature, holders can prove possession of a credential in a manner that is double-spend proof without a ledger. This works for all kinds of redemption use cases like clocking back in at the end of a shift, voting in an election, posting an online review, or redeeming a coupon.

The information we need in any given relationship varies widely with context. Credential exchange protocols must be flexible enough to support many different situations. For example, in You've Had an Automobile Accident, I describe a use case that requires the kinds of ad hoc, messy, and unpredictable interactions that happen all the time in the physical world. Credential exchange readily adapts to these context-dependent, ad hoc situations.

Ease of Mastery

Ease of mastery refers to the capacity of a technology to be easily and broadly adapted and adopted. One of the core features of credential exchange on the identity metasystem is that supports the myriad use cases described above without requiring new applications or user experiences for each one. The digital wallet that is at the heart of credential exchange activities on the self-sovereign internet supports two primary artifacts and the user experiences to manage them: connections and credentials. Like the web browser, even though multiple vendors provide digital wallets, the underlying protocol informs a common user experience.

A consistent user experience doesn’t mean a single user interface. Rather the focus is on the experience. As an example, consider an automobile. My grandfather, who died in 1955, could get in a modern car and, with only a little instruction, successfully drive it. Consistent user experiences let people know what to expect so they can intuitively understand how to interact in any given situation regardless of context.

Accessibility

Accessible technologies are easy to acquire, inexpensive, and resistant to censorship. Because of it's openness, standardization, and support by multiple vendors, credential exchange is easily available to anyone with access to a computer or phone with an internet connection. But we can't limit its use to individuals who have digital access and legal capacity. Ensuring that technical and legal architectures for credential exchange support guardianship and use on borrowed hardware can provide accessibility to almost everyone in the world.

The Sovrin Foundation's Guardianship Working working group has put significant effort into understanding the technical underpinnings (e.g., guardianship and delegation credentials), legal foundations (e.g., guardianship contracts), and business drivers (e.g., economic models for guardianship). They have produced an excellent whitepaper on guardianship that "examines why digital guardianship is a core principle for Sovrin and other SSI architectures, and how it works from inception to termination through looking at real-world use cases and the stories of two fictional dependents, Mya and Jamie."

Self-Sovereign Identity and Generativity

In What is SSI?, I made the claim that SSI requires decentralized identifiers, credential exchange, and autonomy for participants. Dick Hardt pushed back on that a bit and asked me if decentralized identifiers were really necessary? We had a several fun discussions on that topic.

In that article, I unfortunately used decentralized identifiers and verifiable credentials as placeholders for their properties. Once I started looking at properties, I realized that generative identity can't be built on an administrative identity system. Self-sovereign identity is generative not only because of the credential exchange protocols but also because of the properties of the self-sovereign internet upon which those protocols are defined and operate. Without the self-sovereign internet, enabled through DIDComm, you might implement something that works as SSI, but it won't provide the leverage and adaptability necessary to creating a generative ecosystem of uses that creates the network effects needed to propel it to ubiquity.

Our past approach to digital identity has put us in a position where people's privacy and security are threatened by the administrative identity architecture it imposes. Moreover, limiting its scope to authentication and a few unauthenticated attributes, repeated across thousands of websites with little interoperability, has created confusion, frustration, and needless expense. None of the identity systems in common use today offer support for the same kind of ad hoc attribute sharing that happens everyday in the physical world. The result has been anything but generative. Entities who rely on attributes from several parties must perform integrations with all of them. This is slow, complex, and costly, so it typically happens only for high-value applications.

An identity metasystem that supports protocol-mediated credential exchange running on top of the self-sovereign internet solves these problems and promises generative identity for everyone. By starting with people and their innate autonomy, generative identity supports online activities that are life-like and natural. Generative identity allows us to live digital lives with dignity and effectiveness, contemplates and addresses the problems of social inclusion, and supports economic access for people around the globe.

Notes For Alice to prove things about her salary, Attestor would have to include that in the credential they issue to Alice.

Photo Credit: Generative Art Ornamental Sunflower from dp792 (Pixabay)

Tags: ssi identity generative credentials self-sovereign+internet

Monday, 25. January 2021

A Distributed Economy

Comment from Ockam Hello Moved Here

Comment: "I am not part of Ockam, but I've known the folks behind this for awhile. There has been a lot of hair pulling to get to this: https://www.w3.org/TR/did-core/ . This was back in the day. http://manu.sporny.org/2014/credential-based-login/ . A big issue that they still seem to have is data mapping. It happens here: https://www.youtube.com/watch?v=2EP35HO2HVQ&feature=youtu.be [What is

Comment:
"I am not part of Ockam, but I've known the folks behind this for awhile. There has been a lot of hair pulling to get to this:
https://www.w3.org/TR/did-core/ . This was back in the day. http://manu.sporny.org/2014/credential-based-login/ .

A big issue that they still seem to have is data mapping. It happens here: https://www.youtube.com/watch?v=2EP35HO2HVQ&feature=youtu.be [What is a Personal Knowledge Graph- with Ruben Verborgh - The Graph Show]
 and even in the DID space where they rant about interoperability. There is crossover between the SoLiD community and DIDs. They talk about even bigger systems, beyond PDS. In my humble opinion, I believe that there is a blind spot amongst programmers about the wonders about applied category theory. I'm still trying to grasp it myself, but you see it here: https://arxiv.org/abs/1909.04881 [Algebraic Property Graphs], and here: categoricaldata.net/ , and here https://web-cats.gitlab.io/ --> https://arxiv.org/abs/1706.00526 [Knowledge Representation in Bicategories of Relations], and here https://www.youtube.com/watch?v=vnbDmQDvxsE&t=3m41s [ACT 2020 industry showcase]. My feeling is there a white X on the ground that says dig here. It's a reason to learn the maths."

Related to this for Bicatagories of Relations:
Description Logics? https://www.csee.umbc.edu/courses/graduate/691/fall19/07/papers/DescriptionLogicHandbook.pdf



Werdmüller on Medium

The Whole-Employee Professional Development Plan

How I support an employee’s goals beyond their tenure at the company Continue reading on The Startup »

How I support an employee’s goals beyond their tenure at the company

Continue reading on The Startup »


Bill Wendel's Real Estate Cafe

P2P Movement: Reset Real Estate on Consumer Rights

CROWDSOURCING: Fellow real estate consumer advocates and #RETech innovators, miss #REBarCamps, the unconferences that preceded large real estate conferences over a decade, roughly 2006 to… The post P2P Movement: Reset Real Estate on Consumer Rights first appeared on Real Estate Cafe.

CROWDSOURCING: Fellow real estate consumer advocates and #RETech innovators, miss #REBarCamps, the unconferences that preceded large real estate conferences over a decade, roughly 2006 to…

The post P2P Movement: Reset Real Estate on Consumer Rights first appeared on Real Estate Cafe.


Simon Willison

Datasette 0.54, the annotated release notes

Datasette 0.54 is out today, with a number of foundational new features. As I've done for previous releases, here are the annotated release notes with some additional commentary. The _internal database As part of ongoing work to help Datasette handle much larger numbers of connected databases and tables (see Datasette Library) Datasette now maintains an in-memory SQLite database with details

Datasette 0.54 is out today, with a number of foundational new features. As I've done for previous releases, here are the annotated release notes with some additional commentary.

The _internal database

As part of ongoing work to help Datasette handle much larger numbers of connected databases and tables (see Datasette Library) Datasette now maintains an in-memory SQLite database with details of all of the attached databases, tables, columns, indexes and foreign keys. (#1150)

This will support future improvements such as a searchable, paginated homepage of all available tables.

The Datasette Library issue has been open for nearly two years now. It's a need I identified at the NICAR 2019 data journalism conference, where it became apparent that many newsrooms are sat on an enormous pile of data that they have collected but without any central place to keep it all.

Large tech companies usually have some kind of data warehouse or data lake which is meant to address this problem. These are complicated, extremely expensive to run and the ability to add and maintain data in them is often reserved for a high technology priesthood.

I see Datasette as targeting the "small data" end of this equation, where I define small data as anything that will fit on my phone. My iPhone has half a terabyte of storage now!

How many more people and organizations would run their own data warehouse if it could sit on their laptop or run on a $5/month VPS?

You can explore an example of this database by signing in as root to the latest.datasette.io demo instance and then navigating to latest.datasette.io/_internal.

Every commit to Datasette deploys a new demo instance to latest.datasette.io - here's the GitHub Actions workflow. This is useful for all sorts of things - I can use it to test additional devices, remind myself how different pages of Datasette work and in this case I can link to it directly from the documentation.

Plugins can use these tables to introspect attached data in an efficient way. Plugin authors should note that this is not yet considered a stable interface, so any plugins that use this may need to make changes prior to Datasette 1.0 if the _internal table schemas change.

In the ongoing press towards Datasette 1.0 I've decided to start shipping features like this earlier, but with prominent "this is not yet stable" notices to warn people that the details might change. I think this is a better way to preview new features than having them live in a branch or alpha release.

Once I hit 1.0 I'm going to only ship features that are genuinely stable, to avoid having to make any major version number bumps for as long as possible.

Named in-memory database support

As part of the work building the _internal database, Datasette now supports named in-memory databases that can be shared across multiple connections. This allows plugins to create in-memory databases which will persist data for the lifetime of the Datasette server process. (#1151)

The new memory_name= parameter to the Database class can be used to create named, shared in-memory databases.

This exposes a relatively obscure feature of SQLite: it allows in-memory databases to operate with a shared cache, which means multiple database connections can see the same in-memory data.

I can think of so many fun plugin use-cases for this!

A plugin that accepts a URL to a CSV file, then pulls that data into an in-memory table and presents a standard Datasette table interface for interacting with it An API rate-limiting plugin that uses an in-memory database as a persistent store Any plugin that might need to cache something can now do so with its own in-memory database

I haven't used this capability in a plugin myself yet but I'm excited to try it out with something soon.

JavaScript module support

JavaScript modules were introduced in ECMAScript 2015 and provide native browser support for the import and export keywords.

To use modules, JavaScript needs to be included in <script> tags with a type="module" attribute.

Datasette now has the ability to output <script type="module"> in places where you may wish to take advantage of modules. The extra_js_urls option described in Custom CSS and JavaScript can now be used with modules, and module support is also available for the extra_body_script() plugin hook. (#1186, #1187)

This feature was inspired by discussion on this issue about a JavaScript plugin system for Datasette.

I've been getting excited about JavaScript modules recently, partly thanks to Snowpack and Skypack. I enjoyed maintainer Fred K. Schott's GitHub OCTO speaker series talk last week.

The key feature of modules that is useful for Datasette is that they solve the "double imports" problem. Datasette plugins often need to load additional JavaScript libraries, such as Leaflet or React. Using modules means those plugins can, without any knowledge of each other, avoid accidentally loading the same code twice.

datasette-leaflet-freedraw is the first example of a Datasette plugin that takes advantage of the new support for JavaScript modules. See Drawing shapes on a map to query a SpatiaLite database for more on this plugin.

I now have three plugins that load three different versions of Leaflet: datasette-cluster-map, datasette-leaflet-geojson and the new datasette-leaflet-freedraw.

I've started work on a new plugin called datasette-leaflet which will make Leaflet available as a JavaScript module. Those other three plugins can then all depend on datasette-leaflet so that they can share the same library version.

Code formatting with Black and Prettier

Datasette adopted Black for opinionated Python code formatting in June 2019. Datasette now also embraces Prettier for JavaScript formatting, which like Black is enforced by tests in continuous integration. Instructions for using these two tools can be found in the new section on Code formatting in the contributors documentation. (#1167)

I love opinionated code formatting tools. After adopting Black it started to dawn on my quite how much of my programming brain cycles had been spent thinking about indentation - sweating over the details (even in Python) of when to linebreak, how to format complex function definitions and so on.

Switching to Black gave me all of that thinking time back to spend on other things. It even gave me a more productive way to write unit tests.

I've been wanting to do the same thing for JavaScript for years, but there were tiny aspects of the formatting style used by Prettier that I didn't like - specifically the way it formats chained method calls with newlines.

As the positive evidence from using Black continue to build, I realized that the benefits of never having to think about formatting again outweighed any tiny disagreements I might have about aesthetics. So I'm all on board with Prettier now!

Everything else

The rest of the release is mostly small bug fixes.

There are some documentation improvements too. I added a new section to the page about writing tests for plugins on testing outbound HTTP calls with pytest-httpx - any time I find myself having to solve the same problem in more than one plugin is a hint that it should probably be covered by the documentation.

As always, I welcome conversations about the new release (and Datasette in general) on the Datasette discussions forum. I'm also continuing to run Datasette Office Hours every Friday, so if you're interested in a 1-1 conversation about the project please grab a slot!


Jon Udell

The Image of Postgres

At the 2015 Postgres conference, the great IT philosopher Robert r0ml Lefkowitz delivered a talk entitled The Image of Postgres. Here’s the blurb. How do you think about what a database is? If you think of a database as only a place to store your data, then perhaps it does not really matter what the … Continue reading The Image of Postgres

At the 2015 Postgres conference, the great IT philosopher Robert r0ml Lefkowitz delivered a talk entitled The Image of Postgres. Here’s the blurb.

How do you think about what a database is? If you think of a database as only a place to store your data, then perhaps it does not really matter what the internals of that database are; all you really need is a home for your data to be managed, nothing more.

If you think of a database as a place where you develop applications, then your expectations of your database software change. No longer do you only need data management capabilities, but you require processing functions, the ability to load in additional libraries, interface with other databases, and perhaps even additional language support.

If your database is just for storage, there are plenty of options. If your database is your development framework, you need Postgres.

Why? Well, let’s get philosophical.

For over a year, I’ve been using Postgres as a development framework. In addition to the core Postgres server that stores all the Hypothesis user, group, and annotation data, there’s now also a separate Postgres server that provides an interpretive layer on top of the raw data. It synthesizes and caches product- and business-relevant views, using a combination of PL/pgSQL and PL/Python. Data and business logic share a common environment. Although I didn’t make the connection until I watched r0ml’s talk, this setup harkens back to the 1980s when Smalltalk (and Lisp, and APL) were programming environments with built-in persistence. The “image” in r0ml’s title refers to the Smalltalk image, i.e. the contents of the Smalltalk virtual machine. It may also connote reputation, in the sense that our image of Postgres isn’t that of a Smalltalk-like environment, though r0ml thinks it should be, and my experience so far leads me to agree.

I started writing a book to document what I’ve learned and done with this idea. It’s been a struggle to find motivation because, well, being the patron saint of trailing-edge technologies is often lonely and unrewarding. A book on this particular topic is likely to appeal to very few people. Stored procedures? So last century! Yes, Python provides a modern hook, but I can almost guarantee that one of the comments on my first book — “has a vision, but too weird” — would come back around.

I’m tempted not to bother. Maybe I should just focus on completing and polishing the things the book would describe.

And yet, it’s hard to let go. This isn’t just a compelling idea, it’s delivering excellent results. I rewatched r0ml’s talk today and got fired up again. Does it resonate for you? Would you like to see the ideas developed? If you watch the talk, please let me know.

Here are some excerpts to pique your interest.

On databases vs file systems:

I submit that the difference between the database and a file system is that database is a platform for enforcing your business rules.

On ontology:

client: The business guys are always upset because they want to know how many customers we have and we can’t tell them.

r0ml: That doesn’t sound like a very hard problem. SELECT * from the customer table, right?

client: No you don’t understand the problem.

r0ml: OK, what’s the problem?

client: It depends what you mean by customer because if you’re selling cell phone insurance, is the customer the person who has the cell phone account? What if they have two handsets and they’re both insured? What if it’s a family account and there are kids on the plan, do they count as customers? What if it’s a business account and you have 1000 people covered but only 700 using?

r0ml: How my customers you have, that’s a business rule.

So figuring out what your schema is, and figuring out how you organize the stuff and what do you do in the database, that’s all part of enforcing your business rules.

You have to decide what these things mean.

It’s an ontological problem.

You have to classify your knowledge and then enforce your business rules.

On n-tier architecture:

Let us think about the typical web application architecture. This architecture is called the three-tier architecture because it has four tiers. You have your browser, your web server, the thing that runs Python or PHP or JavaScript or Ruby or Java code, and then the database. And that’s always how you do it. And why do you do it that way? Well because that’s how everybody does it.

On Smalltalk and friends:

This is the BYTE magazine cover from August of 1981. In the 70s and the 80s, programming languages had this sort of unique perspective that’s completely lost to history. The way it worked: a programming environment was a virtual machine image, it was a complete copy of your entire virtual machine memory and that was called the image. And then you loaded that up and it had all your functions and your data in it, and then you ran that for a while until you were sort of done and then you saved it out. And this wasn’t just Smalltalk, Lisp worked that way, APL worked that way, it was kind of like Docker only it wasn’t a separate thing because everything worked that way and so you didn’t worry very much about persistence because it was implied. If you had a programming environment it saved everything that you were doing in the programming environment, you didn’t have to separate that part out. A programming environment was a place where you kept all your data and business logic forever.

So then Postgres is kind of like Smalltalk only different.

What’s the difference? Well we took the UI out of Smalltalk and put it in the browser. The rest of it is the same, so really Postgres is an application delivery platform, just like we had back in the 80s.

Sunday, 24. January 2021

Werdmüller on Medium

Here’s what I earned from my tech career

A history of not quite making bank Continue reading on Medium »

A history of not quite making bank

Continue reading on Medium »


Simon Willison

Drawing shapes on a map to query a SpatiaLite database (and other weeknotes)

This week I built a Datasette plugin that lets you query a database by drawing shapes on a map! datasette-leaflet-freedraw SpatiaLite is a SQLite extension that adds a plethora of geospatial querying features. Datasette works well with SpatiaLite already, but every now and then I dip in to a geospatial project to see if there are any neat ways I can extend Datasette's spatial querying support.

This week I built a Datasette plugin that lets you query a database by drawing shapes on a map!

datasette-leaflet-freedraw

SpatiaLite is a SQLite extension that adds a plethora of geospatial querying features. Datasette works well with SpatiaLite already, but every now and then I dip in to a geospatial project to see if there are any neat ways I can extend Datasette's spatial querying support.

datasette-leaflet-freedraw is my new plugin which brings the excellent FreeDraw Leaflet module to Datasette.

It's a pure-JavaScript plugin which looks for Datasette SQL parameters called :freedraw and enhances them with an interactive map. You can then draw on the map to populate the form field with a GeoJSON MultiPolygon representing the shape that you drew. Submit that to the server to excute it as part of a query.

This is best illustrated with a demo. Here's a SQL query that let you draw a shape to find California protected areas (parks, nature reserves and suchlike) that intersect with the polygon that you draw:

select AsGeoJSON(geometry), * from CPAD_2020a_SuperUnits where Intersects(GeomFromGeoJSON(:freedraw), geometry) = 1 and CPAD_2020a_SuperUnits.rowid in ( select rowid from SpatialIndex where f_table_name = 'CPAD_2020a_SuperUnits' and search_frame = GeomFromGeoJSON(:freedraw) )

This uses the California Protected Areas Database by GreenInfo Network, discussed here previously.

Try the SQL query here.

(That from SpatialIndex sub-select at the end is a pattern for taking advantage of SpatiaLite spatial indexes.)

The plugin itself is pretty simple: it scans for freedraw input fields and enhances them with a map interface that writes GeoJSON back to the form field.

You can then use SpatiaLite functions such as GeomFromGeoJSON() to transform that user input into something that can be used in a query.

Another fun query: let's add WHERE PARK_NAME like '%mini%' to the query to filter for just the "mini parks" in San Francisco:

select AsGeoJSON(geometry), * from CPAD_2020a_SuperUnits where PARK_NAME like '%mini%' and Intersects(GeomFromGeoJSON(:freedraw), geometry) = 1 and CPAD_2020a_SuperUnits.rowid in ( select rowid from SpatialIndex where f_table_name = 'CPAD_2020a_SuperUnits' and search_frame = GeomFromGeoJSON(:freedraw) )

Here are the mini parks for San Francisco and for Oakland.

This demo uses the user's shapes as part of a SELECT query, but it's also possible to use Datasette's Writable canned queries to create forms that will save the shapes to a database using an INSERT or UPDATE query.

If you want to try this plugin out note that it currently requires the Datasette 0.54a alpha. This is because it takes advantage of the new <script type="module"> support I added last week. Update 25th January 2021: Datasette 0.54 has now been released.

Other releases this week

I recorded my talk for FOSDEM - and in doing so I broke my unproductive streak and pushed out releases for a whole bunch of Datasette projects - partly through fixing bugs I spotted while trying to record the talk!

Here's the full list, with some annotations.

datasette-auth-github: 0.13 - 2021-01-24
Datasette plugin that authenticates users against GitHub. This is a big release: I finally finished upgrading the plugin to work against Datasette's new-ish authentication and permissions mechanism. datasette-leaflet-freedraw: 0.1.4 - 2021-01-24
Draw polygons on maps in Datasette. See above. datasette-cluster-map: 0.16 - 2021-01-23
Datasette plugin that shows a map for any data with latitude/longitude columns. Fixed some bugs related to the display of marker popups that included foreign key references to other tables. asgi-csrf: 0.8 - 2021-01-23
ASGI middleware for protecting against CSRF attacks. I fixed a broken test caused by my use of an undocumented and unstable API in httpx - details here. I also added a new always_set_cookie=True option for cases where you always want to ensure a CSRF cookie has been set, see #16. datasette-leaflet-geojson: 0.7 - 2021-01-20
Datasette plugin that replaces any GeoJSON column values with a Leaflet map. I fixed a long-standing point of concern: this plugin renders a map for every row on a page containing GeoJSON, which can potentially mean hundreds of maps for queries that return may results. The plugin now renders just the first ten maps and provides a grey clickable "Click to show map" widget for anything past the first ten. datasette: 0.54a0 - 2021-01-19
Lots of stuff in this one, see the release notes. I pushed out an alpha so I could start using the JavaScript modules stuff in datasette-leaflet-freedraw. sqlite-transform: 0.3.1 - 2021-01-18
Tool for running transformations on columns in a SQLite database. I improved the --help. swarm-to-sqlite: 0.3.3 - 2021-01-18
Create a SQLite database containing your checkin history from Foursquare Swarm. I fixed a bug caused by new fields showing up in the Swarm JSON output. sqlite-utils: 3.3 - 2021-01-18
Python CLI utility and library for manipulating SQLite databases. I added the alter=True argument to the .m2m() method, which I needed to fix the bug in swarm-to-sqlite. TIL this week Bulk fetching repository details with the GitHub GraphQL API Fixing broken text encodings with sqlite-transform and ftfy Syntax highlighting Python console examples with GFM Embedding JavaScript in a Jupyter notebook

reb00ted

In praise of incompetence

Shortly after the 2016 election, I pulled out a book written by my grandfather, chronicling the history of Bachhagel, the village in southern Germany where he grew up. I re-read the chapters describing how the Nazis, in short order, took over life there in 1933. His eye-witness account describes in fascinating, and horrifying detail, how quickly the established order and century-old traditions w

Shortly after the 2016 election, I pulled out a book written by my grandfather, chronicling the history of Bachhagel, the village in southern Germany where he grew up.

I re-read the chapters describing how the Nazis, in short order, took over life there in 1933. His eye-witness account describes in fascinating, and horrifying detail, how quickly the established order and century-old traditions were hollowed out and then overrun.

Bachhagel at the time was a tiny place, probably less than 1000 people, out in the countryside, of no political or economic importance. I could have understood how the Nazis would concentrate on the major population and economic centers to crush the opposition, but Bachhagel certainly was as far away from that as possible.

Nevertheless it just took a few months, after which the established order had been swept out and the thugs were fully in charge, day to day, from school to church to public events, and their entire worldview was the only thing that mattered.

With Joe Biden in the office this week, it seems we have turned a chapter. And looking back to the 2016 election day, I realize that although the past four years were bad, people died, children got separated, and many other outrages, we have been lucky. In 2016, I had been expecting worse, and possibly much worse.

Why didn’t it turn out as bad as I had feared? It’s not that the defenders of the republic did a particularly good job. Instead, the would-be usurpers just sucked at getting anything done, including just actually using the power in their. hands. If it had been the original Nazis, the consequences would have been so much worse.

I vastly prefer better defenses, however, than being lucky with having an incompetent opponent. In computer security terms, Trump was a Zero Day Vulnerability of the constitutional system of the US – a successful attack vector that previously had not been known.

Unfortunately, people still aren’t taking this attack vector as a seriously as they should, otherwise we’d have specific legal and practical fix proposals all over the news, which we don’t. Which means the vulnerability remains, and our primary defense will remain the same: hoping that the attacker is incompetent. As long as we don’t fix the system, the next attacker is going to try a similar route and they may very well be more capable. In which case we’d really be in trouble.

So: I raise my glass to imcompetence. Next time, may we get a similar bunch of incompetents. Or actually get our act together and make sure there won’t be a next time.

Friday, 22. January 2021

Rebecca Rachmany

Group Currency: What if you could only transact as a community?

Sufficiency Currency: What Communities Want Starting out with some assumptions, I returned from 11 weeks of travel including visits to 7 intentional communities (ecovillages) with a more solidified idea of what the “sufficiency currency” might look like. Before I go into that, it’s useful for met o distinguish the major differences between the Sufficiency Currency project and other projects. By
Sufficiency Currency: What Communities Want

Starting out with some assumptions, I returned from 11 weeks of travel including visits to 7 intentional communities (ecovillages) with a more solidified idea of what the “sufficiency currency” might look like. Before I go into that, it’s useful for met o distinguish the major differences between the Sufficiency Currency project and other projects.

By the way, we aren’t even sure that “Currency” is the right name for the project, and you can check out this blog for a discussion of different names for the project.

Sufficiency Currency Inquiry

The Sufficiency Currency project is looking at non-monetary currency solutions and the two major inquiries are:

How can we create “group” measures and currencies. Production is a group activity, but money is an individual measure. What if we could only transact as a group or community? What would that type of communications would represent the complexity of interactions among communities? If the purpose of an economy is to provide people’s basic survival needs (food, shelter, energy, health), what would we measure such that we can support everyone and increase the capacity of a society to grow to support more people?

We assert that market economies are not appropriate for support systems, and that we should use a system of pooling for essential services. The idea of pooling asserts that when there is a visible shared pool, people don’t let other people starve, and that if there isn’t enough to go around, the group will be inclined towards group problem-solving.

Goals and Hypothesis

We assert that market economies are not appropriate for support systems, and that we should use a system of pooling for essential services. The idea of pooling asserts that when there is a visible shared pool, people don’t let other people starve, and that if there isn’t enough to go around, the group will be inclined towards group problem-solving.

The fundamental goals of the project are:

Create a form of economy that looks directly at the sustainability of life in the economy, rather than a proxy measure (money). Create local self-sufficiency for communities, and in particular, encourage local regenerative and healthy forms of food and energy production. Specifically for the regenerative/intentional/ecovillage/permaculture movements, support three outcomes: Make it easy to transact within the network as well as with bodies outside of the network. Make it easy to expand the movement and better share the resources coming into the network. For people who want to join or create regenerative communities, make it easier to do so through this unified economic structure.

The initial hypothesis for the Sufficiency Currency is that there are three important measures that a group of communities would want to look at in order to meet the

Market Research: How Does It Really Look?

In September-November of 2020, I traveled to a number of Ecovillages in Italy, Slovenia and Spain, to discuss my vision and understand what they really need in terms of their interactions among themselves and with their neighboring communities who may or may not be ecovillages.

The first thing I realized was that there are quite a few community currency and cryptocurrency projects trying to push their ideas into ecovillages. There are some isolated examples of successful community currency attempts within the intentional community movement, and dozens, if not hundreds of failure stories. The communities themselves are quite aware that they don’t need an alternative monetary currency to function. They are also aware that using monetary and trade based currency doesn’t reflect their values. Finally, these people are busy and they don’t have a lot of spare time for currency experiments. Usually there are one or two people who are responsible for anything that would require a computer. Others have computers and phones, but they are highly disinterested in activities that would require any serious amount of time in front of a screen.

The most gratifying find was that the kind of pooling proposed in the Sufficiency Currency project does appeal to the ecovillages. It would have to be very easy to manage, but they are already managing multiple interactions in their vicinity, and systematizing that is of interest. In fact, the same kind of thinking process has come from some of the national ecovillage support networks, but it hasn’t been a priority, nor is it really within the core competencies of the coordination networks for intentional communities.

All of the communities have some form of trade with the other communities in the area. Depending on the location, they might be interacting with local farms and businesses, cooperatives, or other ecovillages. The agreements look different among different entities, but in general there is a looser type of trade than you would see between businesses. For example, at one ecovillage the nearby town flooded and they went down to help out, taking the volunteers with them and neglecting their harvesting work for a couple of days. There was no formal trade — it was just helping people out — and at the same time, they know that this will be helpful for them when it comes to their needs vis-à-vis the municipal government. Similarly, they had an informal agreement with a local agricultural cooperative, where they were helping with the farm work in return for some produce, but they were giving it a try for a year before they came to any formal agreement. The assumption of cooperation and reciprocation was more important than the specifics of the deal.

Designing the Currency

Designing the “representables”, or currency, for the communities is like designing any product. The first step is to get clear on the problems the community wants to solve. The way they described their problems were mostly in terms of overwhelm:

Every contract needs to be negotiated separately and the documentation is scattered. The negotiation takes a lot of time. Different parties have different ideas of fairness, and fairness isn’t necessarily expressed in monetary value. Some items are truly scarce, while some are abundant, and monetary exchange doesn’t help them identify that type of value or seek solutions. Depending on the type of organization, the terms of business are different. If it’s another ecovillage, it’s a very different type of relationship than if it’s just a nearby village that doesn’t belong to the regenerative agriculture movement.

When translated into measurable currencies that can be represented in software, the Sufficiency Currency would aim, firstly, to create an easy-to-use interface for putting together contacts between two entities. Secondly, the dashboard would include the following measures:

Fairness. Although fairness is subjective, it’s probably the most important measure to maintain for long-term relationships. For a sustainable network of ecovillages, it’s important for the members to feel the other group is dealing fairly with them. Scarcity and abundance. Some items are scarce, for example, the number of trucks that are available. A good representation might be dark or light colors, or thickness of a line. Others are abundance, for example, squash in the summer. Even if the trade is fair, some things might wear out over time while others are easily replenished. Representing the scarcity or abundance of something in the communities allows people to identify joint problems to solve. For example, if there aren’t enough trucks, perhaps they would train some community members to repair and assemble trucks from spare parts. One of the main functions of the currency is to help the communities take joint action. Reputation. Reputation would be a multi-dimensional measure that allows communities to get information about one another, such as how ecologically conscious one community is, whether they have traded fairly in the past, etc. The reputation measures need to be developed over time.

The three measures above are a start for creating an alternative to monetary trading. The goal over time will be to have the communities simply share both their resources and their challenges to grow over time as a movement.


Werdmüller on Medium

Pulmonary fibrosis and me

The moment that gave me back the rest of my life Continue reading on Medium »

The moment that gave me back the rest of my life

Continue reading on Medium »


Simon Willison

Quoting Apple Daily

Tuesday’s chaos arose after China Railway Shenyang failed to deactivate Flash in time, leading to a complete shutdown of its railroads in Dalian, Liaoning province. Staffers were reportedly unable to view train operation diagrams, formulate train sequencing schedules and arrange shunting plans. Authorities fixed the issue by installing a pirated version of Flash at 4:30 a.m. the following day.

Tuesday’s chaos arose after China Railway Shenyang failed to deactivate Flash in time, leading to a complete shutdown of its railroads in Dalian, Liaoning province. Staffers were reportedly unable to view train operation diagrams, formulate train sequencing schedules and arrange shunting plans.

Authorities fixed the issue by installing a pirated version of Flash at 4:30 a.m. the following day.

Apple Daily

Thursday, 21. January 2021

MyDigitalFootprint

Mapping Consumer Financial Services through a new lens!

Unpacking and explaining the Peak Paradox model is here, you will need this backgrounder to understand the model so that this article will make sense; it is a 5-minute read. A new way of seeing the same thing will mean we can act differently; this is the peak paradox model’s core tenet.  ---- A recurring question on an executives mind in the finance industry is; “Why does up or cross-selli

Unpacking and explaining the Peak Paradox model is here, you will need this backgrounder to understand the model so that this article will make sense; it is a 5-minute read. A new way of seeing the same thing will mean we can act differently; this is the peak paradox model’s core tenet. 

----

A recurring question on an executives mind in the finance industry is; “Why does up or cross-selling not deliver in the way we forecast or predict?”  Cross-selling is foundational hypnosis for growth.  H0; this customer has one product, and therefore they should want my other products; all customers need all my core finance products.  With years of data, it is evident that a customer who uses my payment system does not want my other products, and after years of cross-selling we still only have 30% of customers with all products. Whilst we can conclude there is a problem with the hypothesis, we choose to ignore this fact and continue to try to upsell as we don’t have a framework to explore why! If we try a different marketing message, this one will work (what was it about repeating the same thing and expecting a different outcome?) 

This article maps core consumer financial services offerings of Payment (spending), Saving (available, current, on-demand and surplus), Borrowings (mortgage, credit, debt, overdraft, loans) and Investment (long-term growth and or income) onto the Peak Paradox model.  

I will unpack a delta between what a bank or financial service providers products say about their products’ purpose in marketing and terms; and the consumers’ purpose for the same products.  When mapped, it is evident that cross-selling will only work in a limited capacity, and in the majority of cases, there is a misalignment. 

On paper, and with years of data, our upselling hypothesis appears to be misguided as the utopia for growth.  The market has introduced a plethora of new banks,  neo-banks, open banking platforms, banking 1.0, 2.0, 3.0, 4.0 and more; promising digital products for a new next generation. Each new company has a model that attracts customers from accepting the tempting offers from existing providers and then upselling.   We will apply the Peak Paradox model on this perplexing and thorny topic because when we can see something through a different lens, we may decide to act differently, as we have new insights.   

Without a doubt, we all have a reason why upselling and cross-selling does not materialise to the level we predict/ want/ desire; but why does a customer then buy a very similar product from a different financial provider, even though we offer it?  Finding the data to confirm that we have a specific, sound and justifiable explainable reason, which is different to others interruptions, leads to much tension, anxiety, stress, argument and disagreements in our teams and enterprises; both on why we miss the targets and what to do about it.  

We are taking the core banking products in turn.  Remember the purpose here is to provide a non-conformational way for your team to look at the same problem with a new perspective. 

Payment (spending available, accessible and current cash resource).  Spending maps across the entire Peak Paradox map.  We need to spend to survive and therefore is essential at Peak Human Purpose as without food and water we don’t tend to last very long.   Spending on luxury goods meets a requirement for being at Peak Individual Purpose where you look after yourself.  Giving gifts to friends, philanthropy and donations to charity moves the coverage to Peak Society Purpose. Finally, we cannot work without payments,  payments get us towards Peak Work Purpose and covering the entire map.

The observation is that, only at this moment-in-time, when you are using your payment service, can you highlight if the payment provider’s values reflect your purpose or if you feel there is a conflict.  Users have started to use different payment mechanisms to reflect an alignment between their use and how they perceive the payment providers own marketing.  But does that allow them to use this payment provider for everything, or do they have two or more to cope with conflicts?  Can the customer hold the conflict of using the same ethical payment platform to gamble and buy adult services as they give to charity?  How does the market, market payment services?

Saving (surplus).  Saving tend not used for day to day survival (yes individuals often find they have to as a reality of our economic mess), therefore saving does not tend to feature towards Peak Human Purposes for most consumers.  However, if savings are for a house and the house is attractive to a mate for reproduction it can be argued that there is a connection to Peak Human Purpose, but is this saving or borrowings (loans).   However, savings are used to further your own individual purpose or help those closest to you, which means there is at least some societal benefit.  (A gift to charity will be in payments, payments on death to a charity by a Will is a different dimension to be explored,  more case-studies will help your team explore this in more detail.]    Whilst not explicit, some savers understand the economics of savings by themselves leads to borrowings (lending) for others, me saving has an indirect benefit for society. Fewer savers may realise that spending savings are a better way of benefiting a societal purpose as it encourages economic activity and growth.   The point is that “saving” is positioned differently on the Peak Paradox model to that of your payment service.  How your company positions these products will have a direct impact on how users perceive them.  Indeed how your competitor and the media report on these products directly affects user positioning.  We seek the delta between your marketing, the markets positions, the media view, and the users’ own opinion. 

Borrowings (mortgage, credit, debt, overdraft, loans) Brownings at a consumer retail banking level, because the terms and conditions are for an individual or a couple who equally take on the responsibility, meaning that Borrowings are for Individual needs more than society.  Since the use of borrowings that a requester has is vetted by the provider, a company’s processes and regulation (compliance) mean that borrowings focus on Peak Individual Purpose.  Borrowings give the person(s) more agency, and one can argue more freedom, but that depends on responsible lending to ensure that levels of debt are not a burden, which sadly is not always the case.  Borrowings to give to someone else still has to be repaid and when you look at the terms and the processes; borrowing to give away is not an acceptable practice - whereas guarantees are.   

Therefore, borrowings have a different position on the Peak Paradox model.   There is a position where lending does support more basic human survival (payday loans), but this creates tension and conflicts in the users of borrowing products between two Peak Purposes.  Debt can also be used as part of a personal guarantee to provide working capital for a business or enterprise.  This means that borrowings are dragged towards Peak Work Purpose, and depending on if the Borrowings are for growth (thriving) or survival creates different tensions again.   Consider, customers are not buying a specific product; they are buying a generic idea.  When these ideas become confused, it creates tensions.  We might like to give them cool marketing names to deceive ourselves to what is being offered, but it is quite evident in the terms. 

Investment (long-term growth and or income from the investment - capital at risk).  The final bucket considered here. Pre-crypto, climate and ethical investing; investment occupied an area between Peak Individual Purpose and Peak Work Purpose, with variance in risk and return creating an area.  As real-time trading led to no responsibility for the shareholder, the area has shifted towards Peak Individual Purpose.  Crypto, angel, seed and start-up investing has pushed the upper boundary even further to Peak Individual.  However, social and impact investing (labelled here as ESG for convenience) has created a broader and wider market. Those who seek more ethical ideals that align with their own position of the map means the investment is also firmly heading into Peak Society Purpose.  Such diversity.   The same questions need to be reflected on: how is the positioning of investment products aligned to the individual’s need and purpose? Is there a gap between brand, marketing and overall positioning?  


What do we learn? 

Financial services are a very cluttered landscape. Just offering slicker, quicker, faster, less paperwork, and sexy digital services are avoiding the very core of the problem that the Peak Paradox model exposes.  Yes, we can play with terms and marketing. We can create great names, become ever more personalised and seemingly more differentiated in niche services, but fundamentally avoid the conflicts and tensions this creates. 

Innovation.  A term on the lips of every board, executive and management personnel in the finance industry.  We have to be more innovative.  We have clearly been more “clever” in how we bundle, describe, consider risk and differentiate our core products. Still, we appear not to have aligned our corporate purpose, with our products’ purpose with the customers’ purpose for financial services.   Perhaps we should look at innovation in this area. 

Positioning. Suppose we attracted a customer with a faster payment method for essential services, positioned towards Peak Human Purpose and Peak Social Purpose. Why would that customer naturally consider other series until you have either developed trust or re-educated them on your brand position?  Fabulous branding for new customers’ attraction may generate the numbers and look good in the business plan until you need that same “easy to attract customer” to buy something else. 

This is tricky as it might mean rewriting marketing and positioning, and will the marketing/branding team understand this, given that it could affect their KPI’s and bonus?    Such actions also require time and reflection, always the most precious things in an early-stage growth company, as far too many jobs to be done.   

There a delta between an individual consumers perspective of the financial products they are using, the marketing position/ branding you offer and different core products align to other areas meaning that there is unlikely to be a natural cross-selling opportunity, with one exception. There is an alignment of all the products focussed at Peak Individual Purpose. Maybe that is why High Net Wealth (HNW) teams, wealth management and those looking after the ultra-wealthy in banks appear to have a very successful, and aligned business. 

Trust. This asks how to explore the alignment between your product's purpose and the consumer purpose and how this correlates to the “trust” in your brand. If there is a high R2 does it lead to a propensity to utilise more than one product from a financial institute? 


We need to add pensions, tax, gifts, inheritance, B2B, business, corporate and many other financial services to complete the picture, and then the role of the regulator and whose purpose they are working to protect! Anyone up for mapping one company's financial products?



Werdmüller on Medium

How to startup like a bro

The complete guide to crushing it Continue reading on Medium »

The complete guide to crushing it

Continue reading on Medium »

Wednesday, 20. January 2021

Werdmüller on Medium

Do No Harm

It’s been the guiding rule for my entire career. But I was applying it wrong. Continue reading on The Startup »

It’s been the guiding rule for my entire career. But I was applying it wrong.

Continue reading on The Startup »

Tuesday, 19. January 2021

Doc Searls Weblog

Toward new kinds of leverage

“Give me a lever long enough and a fulcrum on which to place it, and I shall move the world,” Archimedes is said to have said. For almost all of the last four years, Donald Trump was one hell of an Archimedes. With the U.S. presidency as his lever and Twitter as his fulcrum, the […]

“Give me a lever long enough and a fulcrum on which to place it, and I shall move the world,” Archimedes is said to have said.

For almost all of the last four years, Donald Trump was one hell of an Archimedes. With the U.S. presidency as his lever and Twitter as his fulcrum, the 45th President leveraged an endless stream of news-making utterances into a massive following and near-absolute domination of news coverage, worldwide. It was an amazing show, the like of which we may never see again.

Big as it was, that show ended on January 8, when Twitter terminated the @RealDonaldTrump account. Almost immediately after that, Trump was “de-platformed” from all these other services as well: PayPal, Reddit, Shopify, Snapchat, Discord, Amazon, Twitch, Facebook, TikTok, Google, Apple, Twitter, YouTube and Instagram. That’s a lot of fulcrums to lose.

What makes them fulcrums is their size. All are big, and all are centralized: run by one company. As members, users and customers of these centralized services, we are also at their mercy: no less vulnerable to termination than Trump.

So here is an interesting question: What if Trump had his own fulcrum from the start? For example, say he took one of the many Trump domains he probably owns (or should have bothered to own, long ago), and made it a blog where he said all the same things he tweeted, and that site had the same many dozens of millions of followers today? Would it still be alive?

I’m not sure it would. Because, even though the base protocols of the Internet and the Web are peer-to-peer and end-to-end, all of us are dependent on services above those protocols, and at the mercy of those services’ owners.

That to me is the biggest lesson the de-platforming of Donald Trump has for the rest of us. We can talk “de-centralization” and “distribution” and “democratization” along with peer-to-peer and end-to-end, but we are still at the mercy of giants.

Yes, there are work-arounds. The parler.com website, de-platformed along with Trump, is back up and, according to @VickerySec (Chris Vickery), “routing 100% of its user traffic through servers located within the Russian Federation.” Adds @AdamSculthorpe, “With a DDos-Guard IP, exactly as I predicted the day it went offline. DDoS Guard is the Russian equivalent of CloudFlare, and runs many shady sites. RiTM (Russia in the middle) is one way to think about it.” Encrypted services such as Signal and Telegram also provide ways for people to talk and be social. But those are also platforms, and we are at their mercy too.

I bring all this up as a way of thinking out loud toward the talk I’ll be giving in a few hours (also see here), on the topic “Centralized vs. Decentralized.” Here’s the intro:

Centralised thinking is easy. Control sits on one place, everything comes home, there is a hub, the corporate office is where all the decisions are made and it is a power game.

Decentralised thinking is complex. TCP/IP and HTTP created a fully decentralised fabric for packet communication. No-one is in control. It is beautiful. Web3 decentralised ideology goes much further but we continually run into conflicts. We need to measure, we need to report, we need to justify, we need to find a model and due to regulation and law, there are liabilities.

However, we have to be doing both. We have to centralise some aspects and at the same time decentralise others. Whilst we hang onto an advertising model that provides services for free we have to have a centralised business model. Apple with its new OS is trying to break the tracking model and in doing so could free us from the barter of free, is that the plan which has nothing to do with privacy or are the ultimate control freaks. But the new distributed model means more risks fall on the creators as the aggregators control the channels and access to a model. Is our love for free preventing us from seeing the value in truly distributed or are those who need control creating artefacts that keep us from achieving our dreams? Is distributed even possible with liability laws and a need to justify what we did to add value today?

So here is what I think I’ll say.

First, we need to respect the decentralized nature of humanity. All of us are different, by design. We look, sound, think and feel different, as separate human beings. As I say in How we save the world, “no being is more smart, resourceful or original than a human one. Again, by design. Even identical twins, with identical DNA from a single sperm+egg, can be as different as two primary colors. (Examples: Laverne Cox and M.LamarNicole and Jonas Maines.)”

This simple fact of our distributed souls and talents has had scant respect from the centralized systems of the digital world, which would rather lead than follow us, and rather guess about us than understand us. That’s partly because too many of them have become dependent on surveillance-based personalized advertising (which is awful in ways I’ve detailed in 136 posts, essays and articles compiled here). But it’s mostly because they’re centralized and can’t think or work outside their very old and square boxes.

Second, advertising, subscriptions and donations through the likes of (again, centralized) Patreon aren’t the only possible ways to support a site or a service. Those are industrial age conventions leveraged in the early decades of the digital age. There are other approaches we can implement as well, now that the pendulum is started to swing back from the centralized extreme. For example, the fully decentralized EmanciPay. A bunch of us came up with that one at ProjectVRM way back in 2009. What makes it decentralized is that the choice of what to pay, and how, is up to the customer. (No, it doesn’t have to be scary.) Which brings me to—

Third, we need to start thinking about solving business problems, market problems, technical problems, from our side. Here is how Customer Commons puts it:

There is … no shortage of of business problems that can only be solved from the customer’s side. Here are a few examples :

Identity. Logins and passwords are burdensome leftovers from the last millennium. There should be (and already are) better ways to identify ourselves, and to reveal to others only what we need them to know. Working on this challenge is the SSI—Self-Sovereign Identity—movement. The solution here for individuals is tools of their own that scale. Subscriptions. Nearly all subscriptions are pains in the butt. “Deals” can be deceiving, full of conditions and changes that come without warning. New customers often get better deals than loyal customers. And there are no standard ways for customers to keep track of when subscriptions run out, need renewal, or change. The only way this can be normalized is from the customers’ side. Terms and conditions. In the world today, nearly all of these are ones companies proffer; and we have little or no choice about agreeing to them. Worse, in nearly all cases, the record of agreement is on the company’s side. Oh, and since the GDPR came along in Europe and the CCPA in California, entering a website has turned into an ordeal typically requiring “consent” to privacy violations the laws were meant to stop. Or worse, agreeing that a site or a service provider spying on us is a “legitimate interest.” Payments. For demand and supply to be truly balanced, and for customers to operate at full agency in an open marketplace (which the Internet was designed to be), customers should have their own pricing gun: a way to signal—and actually pay willing sellers—as much as they like, however they like, for whatever they like, on their own terms. There is already a design for that, called Emancipay. Internet of Things. What we have so far are the Apple of things, the Amazon of things, the Google of things, the Samsung of things, the Sonos of things, and so on—all silo’d in separate systems we don’t control. Things we own on the Internet should be our things. We should be able to control them, as independent customers, as we do with our computers and mobile devices. (Also, by the way, things don’t need to be intelligent or connected to belong to the Internet of Things. They can be, or have, picos.) Loyalty. All loyalty programs are gimmicks, and coercive. True loyalty is worth far more to companies than the coerced kind, and only customers are in position to truly and fully express it. We should have our own loyalty programs, to which companies are members, rather than the reverse. Privacy. We’ve had privacy tech in the physical world since the inventions of clothing, shelter, locks, doors, shades, shutters, and other ways to limit what others can see or hear—and to signal to others what’s okay and what’s not. Instead, all we have are unenforced promises by others not to watching our naked selves, or to report what they see to others. Or worse, coerced urgings to “accept” spying on us and distributing harvested information about us to parties unknown, with no record of what we’ve agreed to. Customer service. There are no standard ways to call for service yet, or to get it. And there should be. Advertising. Our main problem with advertising today is tracking, which is failing because it doesn’t work. (Some history: ad blocking has been around since 2004, it took off in 2013, when the advertising and publishing industries gave the middle finger to Do Not Track, which was never more than a polite request in one’s browser not to be tracked off a site. By 2015, ad blocking alone was the biggest boycott i world history. And in 2018 and 2019 we got the GDPR and the CCPA, two laws meant to thwart tracking and unwanted data collection, and which likely wouldn’t have happened if we hadn’t been given that finger.) We can solve that problem from the customer side with intentcasting,. This is where we advertise to the marketplace what we want, without risk that our personal data won’t me misused. (Here is a list of intentcasting providers on the ProjectVRM Development Work list.)

We already have examples of personal solutions working at scale: the Internet, the Web, email and telephony. Each provides single, simple and standards-based ways any of us can scale how we deal with others—across countless companies, organizations and services. And they work for those companies as well.

Other solutions, however, are missing—such as ones that solve the eight problems listed above.

They’re missing for the best of all possible reasons: it’s still early. Digital living is still new—decades old at most. And it’s sure to persist for many decades, centuries or millennia to come.

They’re also missing because businesses typically think all solutions to business problems are ones for them. Thinking about customers solving business problems is outside that box.

But much work is already happening outside that box. And there already exist standards and code for building many customer-side solutions to problems shared with businesses. Yes, there are not yet as many or as good as we need; but there are enough to get started.

A lot of levers there.

For those of you attending this event, I’ll talk with you shortly. For the rest of you, I’ll let you know how it goes.


Werdmüller on Medium

Ma

The silver lining of the pandemic was getting to care for her. Continue reading on The Shadow »

The silver lining of the pandemic was getting to care for her.

Continue reading on The Shadow »


Bill Wendel's Real Estate Cafe

#MLKDay2021 – I Have a Post-Pandemic Dream

Earlier today, a real estate colleague shared this liberating example of #WFH – Work From Home, a futuristic concept car from Nissan.   https://youtu.be/qSyMUr7cRXg It’s… The post #MLKDay2021 - I Have a Post-Pandemic Dream first appeared on Real Estate Cafe.

Earlier today, a real estate colleague shared this liberating example of #WFH – Work From Home, a futuristic concept car from Nissan.   https://youtu.be/qSyMUr7cRXg It’s…

The post #MLKDay2021 - I Have a Post-Pandemic Dream first appeared on Real Estate Cafe.

Monday, 18. January 2021

Identity Woman

Podcast: The Domains of Identity and SSI

I was on the UbiSecure Podcast where I talked about The Domains of Identity and SSI. You can also listen to it on  Apple, Google, Spotify etc. The post Podcast: The Domains of Identity and SSI appeared first on Identity Woman.

I was on the UbiSecure Podcast where I talked about The Domains of Identity and SSI. You can also listen to it on  Apple, Google, Spotify etc.

The post Podcast: The Domains of Identity and SSI appeared first on Identity Woman.


Quoted in NYT

I was quoted in this article about Tim Berner’s Lee and the Solid Project. ….“No one will argue with the direction,” said Liam Broza, a founder of LifeScope, an open-source data project. “He’s on the right side of history. But is what he’s doing really going to work?” Others say the Solid-Inrupt technology is only […] The post Quoted in NYT appeared first on Identity Woman.

I was quoted in this article about Tim Berner’s Lee and the Solid Project. ….“No one will argue with the direction,” said Liam Broza, a founder of LifeScope, an open-source data project. “He’s on the right side of history. But is what he’s doing really going to work?” Others say the Solid-Inrupt technology is only […]

The post Quoted in NYT appeared first on Identity Woman.

Sunday, 17. January 2021

Aaron Parecki

The Perfect Remote Control ATEM Mini Interview Kit

This tutorial will walk you through setting up an ATEM Mini Pro kit you can ship to a remote location and then control from your studio. You can use this to ship a remote interview kit to someone where all they have to do is plug in a few connections and you'll be able to control everything remotely!

This tutorial will walk you through setting up an ATEM Mini Pro kit you can ship to a remote location and then control from your studio. You can use this to ship a remote interview kit to someone where all they have to do is plug in a few connections and you'll be able to control everything remotely!

The overall idea is you'll ship out an ATEM Mini Pro (or ISO), a Blackmagic Pocket Cinema Camera 4K (or any other camera, but the Pocket 4K can be controlled by the ATEM!), and a Raspberry Pi. The guest will connect the Raspberry Pi to their home network, connect the camera's HDMI to the ATEM, turn everything on, and you'll immediately be able to control the remote end yourself!

The remote ATEM can then stream to any streaming service directly, or if you have a Streaming Bridge, you can get a high quality video feed of the remote guest brought directly into your studio!

Devices Used in this Tutorial ATEM Mini Pro or ATEM Mini Pro ISO GL.iNet AR750S Travel Router Raspberry Pi 4 (a Pi 3 will also work, but they aren't that much cheaper anyway) a MicroSD Card for the Raspberry Pi, I like the 32gb A2 cards a USB Ethernet adapter to get an additional ethernet port on the Raspberry Pi Blackmagic Pocket Cinema Camera 4K (any camera will do, but the Pocket 4K can be controlled remotely too!) Blackmagic Streaming Bridge to receive the remote ATEM video feed in your studio Set up the Studio Side

We'll start with setting up the studio side of the system. This is where your computer to control the remote ATEM will be, and this end will have the VPN server. If you have a Streaming Bridge, it would be on this end as well in order to receive the streaming video feed from the remote ATEM.

First we're going to set up the GL.iNet router as a Wireguard server.

Plug your computer in to one of the two ethernet ports on the right, or connect to its WiFi hotspot. There's a sticker on the bottom showing the default wifi name and password as well as the IP address of the router. (It's probably 192.168.8.1).

Open that address in your browser and it will prompt you to set up an admin password. Make sure you keep that somewhere safe like a password manager.

Set up Internet Access for the Travel Router

The first thing we need to do is get internet access set up on this router. You'll definitely want to hardwire this in to your studio network rather than use wifi. Plug an ethernet cord from your main router in the studio to the port on the left. Once that's connected, the admin dashboard should pop up a section to configure the wired connection. You can use DHCP to get it on your network. Eventually you'll want to give this device a fixed IP address by going into your main router and setting a static DHCP lease. The specifics of that will depend on your own router so I'll leave that up to you to look up.

In my case the travel router's IP address is 10.10.12.102.

Configure the Wireguard Server

Now we're ready to set up the travel router as a Wireguard server. Wireguard is relatively new VPN software that is a lot faster and easier to use compared to older VPN software like OpenVPN. It's conveniently already built in to the travel router as well, making it an excellent option for this.

Go to the VPN menu on the side and expand it to reveal the Wireguard Server tab.

Click "Initialize Wireguard Server" and you'll be able to set it up. Enable the "Allow Access Local Network" toggle, and change the "Local IP" to 172.16.55.1. (It doesn't really matter what IP range you use for the Wireguard interface, but this address is unlikely to conflict with your existing network.)

Now you can click "Start", and then go into the "Management" tab.

Click the big "Add a New User" button. Give it a name like "RemoteATEM". This will create a config for it which you'll use to set up the remote side.

Click on the icon under "Configurations" and click the "Plain Text" tab.

Copy that text into a text editor (not Word or Google Docs!). We're going to make a few changes to make it look like the below.

[Interface] Address = 172.16.55.2/32 ListenPort = 42170 PrivateKey = <YOUR PRIVATE KEY> [Peer] AllowedIPs = 172.16.55.0/24, 192.168.8.0/24 Endpoint = <YOUR IP ADDRESS>:51820 PersistentKeepalive = 25 PublicKey = <YOUR PUBLIC KEY>

We don't want the ATEM to send the video feed over the VPN, so change the AllowedIPs line to: 172.16.55.0/24, 192.168.8.0/24. If it's set to 0.0.0.0/0 then all the traffic on the remote end will be funneled through your VPN server and your studio's internet connection. That's usually what you want when you're using a VPN for privacy, but we don't want to add latency to sending the video feed if you're streaming from the remote end to YouTube directly. The IP address in the Endpoint = line is the public IP address of your studio network, so make sure you leave that line alone. You can remove the DNS = line since we aren't routing all network traffic through the VPN. Make sure you keep the PrivateKey and PublicKey that your router generated though!

This next part is the magic that makes it work. The key here is we need to let other devices on the WireGuard server end access devices on the LAN side of the WireGuard client. The ATEM will be behind the WireGuard client (described in the next section), the ATEM itself wouldn't normally be visible to other things on the Wireguard server side.

Unfortunately this is the most complicated step. You'll need to edit a text file on the router by connecting to it via ssh. Open Terminal on a mac, or PuTTY on Windows.

ssh root@192.168.8.1

The root ssh password is the same admin password you created when you first set up the router. (You won't see your keystrokes as you're typing your password.)

You'll need a text editor to edit the file. If you're familiar with vi, it's already installed. If you want something easier to use, then you can install nano.

opkg update opkg install nano

Now you can edit the file using nano:

nano /etc/config/wireguard_server

Navigate to the bottom of the file and add this line:

list subnet '192.168.5.0/24'

The file should now look something like the below

config servers option local_port '51820' option local_ipv6 'fd00:db8:0:abc::1' option private_key '<YOUR PRIVATE KEY>' option public_key '<YOUR PUBLIC KEY>' option access 'ACCEPT' option local_ip '172.16.55.1' option enable '1' config peers 'wg_peer_9563' option name 'RemoteATEM' option client_key '<CLIENT PUBLIC KEY>' option private_key '<CLIENT PRIVATE KEY>' option client_ip '172.16.55.2/32' list subnet '192.168.5.0/24'

To save and exit, press control X, and press Y and enter when prompted. You'll need to reboot this device once this is set up, so click the "reboot" button in the top right corner of the admin panel.

Set Up Port Forwarding

The last step on the studio side is to set up port forwarding from your studio router to forward port 51820 to the IP address of the WireGuard server. This is what will let the remote end be able to connect to your studio VPN. How you do this will depend on what router you use in your studio. Most routers will have a web interface to configure them which will let you set up port forwarding rules.

You'll need to know the IP address of your studio router as well as the IP address of the travel router. Create a port forwarding rule to forward port 51820 to your travel router's IP address.

For example, this is what it looks like in my Unifi router to create the forwarding rule.

Alright, we're set on this end! You now have a WireGuard server accessible from outside of your network! If you're curious to keep experimenting with WireGuard, you can even set up your mobile phone or laptop as a WireGuard client so that you can connect back to your studio on the go!

Next, we're ready to set up the remote end for the ATEM Mini Pro.

Set up the Remote Side for the ATEM

The remote kit side will have the ATEM Mini Pro, the camera, and the WireGuard client.

When the WireGuard client powers on, it will connect to your studio network and make the ATEM Mini acessible from your studio network.

I want to preface this section by saying I tried really hard to make this work with another GL.iNet router, but ran into a few blockers before switching to using a Raspberry Pi. As such, this step is a lot more involved and requires quite a bit more command line work than I would like. If you are reading this and happen to know how to do this with the GL.iNet router instead, I would be very interested in hearing it! Feel free to get in touch with details!

Install Raspberry Pi OS

The first thing to do is burn the Raspberry Pi OS image to a MicroSD card. Download the Raspberry Pi Imager app, and go through the steps of installing the Raspberry Pi OS. If you're comfortable on the command line, you can use the "lite" version which doesn't include the desktop environment. I prefer this since I'm never going to be connecting a monitor to this anyway after the first round of setup.

After the card is created, stick it in the Raspberry Pi and turn it on. You'll need a keyboard and monitor connected to set things up the first time. You'll also want to make sure the Raspberry Pi has an internet connection, so plug in an ethernet cord to the ethernet port on the Pi into your home router.

Install WireGuard

It looks like in the near future, WireGuard will be included by default in Raspberry Pi OS, so first check if it's already there by typing which wg, and if it shows you a path like /usr/bin/wg then you can skip this step since it's already installed!

If you need to install WireGuard, you can install it from the "testing" packages by running these commands:

echo "deb http://archive.raspbian.org/raspbian testing main" | sudo tee --append /etc/apt/sources.list.d/testing.list sudo apt update sudo apt install -y wireguard

Now when you run which wg you should see that it's installed.

WireGuard Setup

Remember that big block of text you created in the previous step? It's time to use it! You'll need to paste that into the WireGuard configuration file on the Raspberry Pi.

sudo nano /etc/wireguard/wg0.conf

Paste in the full contents of the configuration replacing anything that is already in that file if any.

[Interface] Address = 172.16.55.2/32 ListenPort = 42170 PrivateKey = <YOUR PRIVATE KEY> [Peer] AllowedIPs = 172.16.55.0/24, 192.168.8.0/24 Endpoint = <YOUR IP ADDRESS>:51820 PersistentKeepalive = 25 PublicKey = <YOUR PUBLIC KEY>

Save and quit by pressing control X, then Y and enter when prompted.

Turn the Raspberry Pi into a Router

The Raspberry Pi will be sharing its internet connection with the ATEM Mini, so it's time to turn it into a router! (This is the reason I wanted to use a travel router instead, because this step is already done in that case.)

First, plug in the USB ethernet adapter to one of the USB ports, and connect an ethernet cable from that to the ATEM Mini.

Now we need to set up that ethernet device. Edit the file using the command below:

sudo nano /etc/network/interfaces.d/eth1

Paste in the following text exactly as written:

auto eth1 iface eth1 inet static address 192.168.5.1 netmask 255.255.255.0

This creates a new network on the Raspberry Pi which will be the private network for the ATEM Mini.

We're going to set up a few more things first, then eventually reboot the Raspberry Pi.

sudo apt update sudo apt install -y isc-dhcp-server firewalld

Now edit the DHCP server config file:

sudo nano /etc/dhcp/dhcpd.conf

You'll need to set the domain-name and domain-name-servers as follows:

option domain-name "rpi.local"; option domain-name-servers 1.1.1.1, 8.8.8.8;

Scroll down to the bottom and add the following lines as well:

subnet 192.168.5.0 netmask 255.255.255.0 { range 192.168.5.100 192.168.5.200; option routers 192.168.5.1; option subnet-mask 255.255.255.0; }

Next, open the /etc/default/isc-dhcp-server file:

sudo nano /etc/default/isc-dhcp-server

Add eth1 to the INTERFACESv4 line and save the file.

INTERFACESv4="eth1" INTERFACESv6=""

Now we can set up the firewall to allow this traffic through and enable IP masquerading.

sudo firewall-cmd --add-service=dhcp --permanent sudo firewall-cmd --add-masquerade --permanent

Lastly, we'll tell the WireGuard client to start up on boot by typing:

sudo systemctl enable wg-quick@wg0

Finally, we're ready to reboot the Raspberry Pi and see if it all works!

sudo reboot

With any luck, the Raspberry Pi will boot up and connect to your WireGuard server and everything will be working!

On the Raspberry Pi, you should be able to type sudo wg to see the status of the WireGuard client. If it worked, you'll see the endpoint you're connected to as well as a "latest handshake" and some transfer stats showing how much data has gone over the VPN.

Let's Try it Out!

With everything up and running, it's time to try to connect to your ATEM! You'll need to find or set the IP address of the ATEM on the remote side. You can try out 192.168.5.100 since it will be the first device that the DHCP server sees, or you can connect your ATEM over USB and use the ATEM Setup app to find the address.

Of course, to really put it to the test, you'll want to unplug from USB and see if you can connect from your laptop through the VPN.

Remember, at this point your laptop running the ATEM Software Control app should be connected to your travel router either over wifi or ethernet. Your ATEM Mini is the remote end, and that's connected to the Raspberry Pi.

Open the ATEM Software Control app and type in the ATEM's IP address under "Manual IP Address". (It won't be able to auto-discover it on the network since it's going over a VPN.)

If you have a Blackmagic camera connected to the ATEM, you'll be able to control it from the Camera tab since it's connected to the ATEM via HDMI! No networking stuff to worry about there.

Using the Streaming Bridge

Now you're ready to choose where to stream! The remote ATEM can stream directly to YouTube if that's how you're using it. Because of the way we set up WireGuard, the ATEM will use the remote end's internet connection rather than sending the video through the VPN and out of your studio. That saves a hop and will give you better performance.

If you want to stream to your Streaming Bridge, that should "just work" by using the local IP address of the Streaming Bridge. Connect the Streaming Bridge to your travel router, then open the ATEM Setup app to find its IP address. In my case it's 192.168.8.127.

Scroll down and choose "Local network with key". It's a "local network" because it's going over the VPN. The nice thing about doing it this way is you don't have to set up port forwarding in your studio's router to the bridge, since it will all happen over the VPN!

Next click the "External ATEM Mini Pro" tab to export the configuration file.

You'll need to make one change in the file since the normal local network discovery that it does won't work over the VPN. Open the xml file in a text editor like TextEdit or Notepad, and replace the URL with the IP address of your Streaming Bridge, for example:

<servers> <server> <name>Default</name> <url>rtmp://192.168.8.127:1935/app</url> </server> </servers>

Now you can load that config file in the ATEM Software Control from the "Stream" menu in the top menu bar, and you'll have a new streaming destination to choose!

Recording Locally

If you also ship a USB drive plugged in to the remote ATEM, you can record the program locally as a backup! If you use a Blackmagic Pocket Cinema Camera 4K, you can even tell the camera itself to record a high quality video to an SD card or SSD in the camera itself!

And that's it! Hopefully all of this worked for you. If you have any trouble, feel free to post a question on the Live Video Tech forums to get some assistance!

Follow me on YouTube for more tips and tricks about livestreaming, video production, and the ATEM Mini!

Friday, 15. January 2021

MyDigitalFootprint

Data: Governance and Geopolitics

Interesting article here from the Center for Strategic and International Studies. Written by Gregory F. Treverton, a senior adviser. “How data is governed can be thought of along several lines of activity: legislating privacy and data use, regulating content, using antitrust laws to dilute data monopolies, self-regulating by the tech giants, regulating digital trade, addressing intellectual pr
Interesting article here from the Center for Strategic and International Studies. Written by Gregory F. Treverton, a senior adviser.

“How data is governed can be thought of along several lines of activity: legislating privacy and data use, regulating content, using antitrust laws to dilute data monopolies, self-regulating by the tech giants, regulating digital trade, addressing intellectual property rights (IPR) infringement, assuring cybersecurity, and practicing cyber diplomacy. Of these, antitrust, regulation, and privacy are most immediately in the spotlight, and are the focus of this commentary, but it will also touch briefly on the connections with other issues.”


I have written a lot that data is data ( and not oil) but this article misses the point about the difference between

Data Governance (micro)

Data Governance (marco)

Governance with Data

Data governance (micro) is a massively important and large field focused on ensuring that the quality, provenance, lineage of data, including bias, ethics, data analysis, statistics and many more topics.  Is ensure that the data that is used and how it is used to make decisions is of the best quality possible.

Data Governance (marco) as per the article is on the border topic of international regulation for data. 

Governance with Data is where the board, those in office and those in positions of authority are given or presented with data (via a data governance process) to make complex judgment.   


The take away “The digital age presents geopolitical and philosophical problems with complexity and speed beyond the reach of the existing global architecture that underrepresents both emerging powers, like China, and ever more influential private sector actors, from the tech giants to the Gates Foundation. We are addressing twenty-first-century problems with a twentieth-century mindset, approach, and toolkit. is the take-away. “ with an eighth-century view of sovereign, governance and oversight.  We have to update our more than just our mindset to embrace this change. We need to upgrade the very nature of understand hierarchy, power, influence, agency and purpose to name a few. 




The World’s Most Influential Values, In One Graphic

Source: https://www.visualcapitalist.com/most-influential-values/ Add this to my principles and rules thinking about how to connect risk frameworks.   The key point is the delta.  Will map this onto Peak Paradox
Source: https://www.visualcapitalist.com/most-influential-values/
Add this to my principles and rules thinking about how to connect risk frameworks.   The key point is the delta.  Will map this onto Peak Paradox







Rethinking the Borad

Interesting article here at DEMYST by Neil Tsappis and Dr Tamara Rusell, presenting principles for a new boardroom operating model.  Worth a read. Worth picking up on several themes 1. diversity at the board - and the assumption that balance is what we want to achieve.  Balance in representation and balance in decisions/ judgment. Compremise may not be the right answer
Interesting article here at DEMYST by Neil Tsappis and Dr Tamara Rusell, presenting principles for a new boardroom operating model.  Worth a read.
Worth picking up on several themes
1. diversity at the board - and the assumption that balance is what we want to achieve.  Balance in representation and balance in decisions/ judgment. Compremise may not be the right answer.


2.  Huristics - assumes we have rules, most boards I would suggest have to continually create new ways of working and understanding, therefore resting back on ways of doing something is not going to get to better decisions.
3. fully agree with the market articulation, faster, quicker and more complex.  But we don't actually question the structure of the board and its role.  It apppers to be sacrosanct.   I have explored a few times Organistion 2.0 - where we question the entire thinking. 
4. It does not address the skills gap in data - this to me is something we have to do.
love their work and makes me think





reb00ted

Are most Facebook users cost centers, rather than profit centers?

According to CNBC, Facebook made $7.89 in revenue per average global user in the 3rd quarter last year (with a high of $39.63 in the US and Canada, and a low of $2.22 outside US, Canada, Europe and Asia-Pacific). According to Yahoo! Finance and my calculation, if its expenses in the same quarter were $13.4 billion, expense per user was $13.4 / $21.5 * $7.89 = $4.92 on average (proportiona

According to CNBC, Facebook made $7.89 in revenue per average global user in the 3rd quarter last year (with a high of $39.63 in the US and Canada, and a low of $2.22 outside US, Canada, Europe and Asia-Pacific).

According to Yahoo! Finance and my calculation, if its expenses in the same quarter were $13.4 billion, expense per user was $13.4 / $21.5 * $7.89 = $4.92 on average (proportionally allocated given expense / revenue ratio).

Revenue per user is obviously quite different in different parts of the world, but what about costs? It seems to me that on a per-user-basis, selling and serving all those ads in the US and Canada that led to so much revenue per user is probably more expensive, compared to some places that have less commerce. But as dramatically different as $39.63 and $2.22 on the revenue side? I don’t think so. Not even close.

In other words, users in the rest of the world at $2.22 of revenue per user are almost certainly not profitable. Even if expenses there were only half of average, it would still not be enough.

Of course these numbers are averages across the regions, and chances are that the differences between users within one region are also quite striking. I don’t have numbers on those. But I would bet that some users in the US and Canada also bring in less revenue than the $4.92 in average cost per user.

Who would those unprofitable users be in the US, say? Well, those demographics and those neighborhoods in the social graph in which advertisers see little opportunities to make a sale, because, for example, everybody is unemployed and angry.

(So if, for example, a certain presidential campaign came by and wanted to specifically target this demographic with political ads … I for one can vividly imagine the leap of joy of some Facebook business guy who finally saw how to get promoted: “I turned a million users from being a cost center to being a profit center”. And democracy be damned. Of course, I’m speculating here, but directionally I don’t think I’m wrong.)

Which suggests another strategy to unseat Facebook as the dominant social network: focus on picking off the users that generate the most revenue for Facebook, as they subsidize the rest. If that relatively small subset of users jumped ship, the rest of the business would become unprofitable.

(I jotting this down because I hadn’t seen anybody suggest this strategy. We do need to find ways of ending surveillance capitalism after all.)

Thursday, 14. January 2021

reb00ted

Decentralization is about authority: who has it, and how can they exercise it.

This an other gems in a great piece by Beaker Browser developer Paul Frazee.

This an other gems in a great piece by Beaker Browser developer Paul Frazee.

Monday, 11. January 2021

Doc Searls Weblog

How we save the world

Let’s say the world is going to hell. Don’t argue, because my case isn’t about that. It’s about who saves it. I suggest everybody. Or, more practically speaking, a maximized assortment of the smartest and most helpful anybodies. Not governments. Not academies. Not investors. Not charities. Not big companies and their platforms. Any of those […]

Let’s say the world is going to hell. Don’t argue, because my case isn’t about that. It’s about who saves it.

I suggest everybody. Or, more practically speaking, a maximized assortment of the smartest and most helpful anybodies.

Not governments. Not academies. Not investors. Not charities. Not big companies and their platforms. Any of those can be involved, of course, but we don’t have to start there. We can start with people. Because all of them are different. All of them can learn. And teach. And share. Especially since we now have the Internet.

To put this in a perspective, start with Joy’s Law: “No matter who you are, most of the smartest people work for someone else.” Then take Todd Park‘s corollary: “Even if you get the best and the brightest to work for you, there will always be an infinite number of other, smarter people employed by others.” Then take off the corporate-context blinders, and note that smart people are actually far more plentiful among the world’s customers, readers, viewers, listeners, parishioners, freelancers and bystanders.

Hundreds of millions of those people also carry around devices that can record and share photos, movies, writings and a boundless assortment of other stuff. Ways of helping now verge on the boundless.

We already have millions (or billions) of them are reporting on everything by taking photos and recording videos with their mobiles, obsolescing journalism as we’ve known it since the word came into use (specifically, around 1830). What matters with the journalism example, however, isn’t what got disrupted. It’s how resourceful and helpful (and not just opportunistic) people can be when they have the tools.

Because no being is more smart, resourceful or original than a human one. Again, by design. Even identical twins, with identical DNA from a single sperm+egg, can be as different as two primary colors. (Examples: Laverne Cox and M. Lamar. Nicole and Jonas Maines.)

Yes, there are some wheat/chaff distinctions to make here. To thresh those, I dig Carlo Cipolla‘s Basic Laws on Human Stupidity (.pdf here) which stars this graphic:

The upper right quadrant has how many people in it? Billions, for sure.

I’m counting on them. If we didn’t have the Internet, I wouldn’t.

In Internet 3.0 and the Beginning of (Tech) History, @BenThompson of @Stratechery writes this:

The Return of Technology

Here technology itself will return to the forefront: if the priority for an increasing number of citizens, companies, and countries is to escape centralization, then the answer will not be competing centralized entities, but rather a return to open protocols. This is the only way to match and perhaps surpass the R&D advantages enjoyed by centralized tech companies; open technologies can be worked on collectively, and forked individually, gaining both the benefits of scale and inevitability of sovereignty and self-determination.

—followed by this graphic:

If you want to know what he means by “Politics,” read the piece. I take it as something of a backlash by regulators against big tech, especially in Europe. (With global scope. All those cookie notices you see are effects of European regulations.) But the bigger point is where that arrow goes. We need infrastructure there, and it won’t be provided by regulation alone. Tech needs to take the lead. (See what I wrote here three years ago.) But our tech, not big tech.

The wind is at our backs now. Let’s sail with it.

Bonus links: Cluetrain, New Clues, World of EndsCustomer Commons.

And a big HT to my old buddy Julius R. Ruff, Ph.D., for turning me on to Cipolla.

[Later…] Seth Godin calls all of us “indies.” I like that. HT to @DaveWiner for flagging it.


reb00ted

And our ICUs are full

Santa Clara County, the heart of Silicon Valley. Better not fall off the ladder or silly things like that for the foreseeable future.

Santa Clara County, the heart of Silicon Valley. Better not fall off the ladder or silly things like that for the foreseeable future.


Hal Plotkin asks for the government to build equity in the technology it uses, and not just rent it

In these times: revolutionary. Fifty years ago it would have been meh because we did indeed get a publicly-owned highway system. Could be as simple as requiring software to become open source some time after a purchase.

In these times: revolutionary. Fifty years ago it would have been meh because we did indeed get a publicly-owned highway system. Could be as simple as requiring software to become open source some time after a purchase.

Saturday, 09. January 2021

Aaron Parecki

3D Printed Modular Cage for Blackmagic Bidirectional SDI/HDMI Converters

This modular cage allows you to stack Blackmagic Bidirectional SDI/HDMI 3G Converters on a desk or behind other gear.

This modular cage allows you to stack Blackmagic Bidirectional SDI/HDMI 3G Converters on a desk or behind other gear.

The design is composed of four parts: the main shelf, the top shelf, a top plate that sits on top of the top shelf, and an optional angled base. The angled base plate lets you mount the whole stack on top of rack gear that's mounted at a 9° angle, such as the StarTech 8U desktop rack. You can print as many of the main shelves as you need to stack.

The shelves are designed so that the converters can slip in from one side with enough of a gap on top to be able to insert and remove them even after a stack is assembled.

If you'd like, you can glue the stack together to make a solid structure. Otherwise the pegs are long enough that the stack is reasonably stable even without glue.

Files bmd_micro3g_shelf.stl bmd_micro3g_top_shelf.stl bmd_micro3g_top.stl bmd_micro3g_base.stl

These designs are licensed under the Creative Commons Attribution license.

Did you make a print of this? Tweet a photo and include the link to this blog post to be featured in the comments below!

Friday, 08. January 2021

Bill Wendel's Real Estate Cafe

Open Letter: Processing assault on the American Dream

Open letter to my real estate peers. Inman News, a well-respected presence in the real estate ecosystem, is acknowledging what happened yesterday and but trying… The post Open Letter: Processing assault on the American Dream first appeared on Real Estate Cafe.

Open letter to my real estate peers. Inman News, a well-respected presence in the real estate ecosystem, is acknowledging what happened yesterday and but trying…

The post Open Letter: Processing assault on the American Dream first appeared on Real Estate Cafe.


reb00ted

Bitcoin is above USD 40,000

I did not think that would happen. At least not for many years.

I did not think that would happen. At least not for many years.


California has a public data broker registry

Maintained by the state’s Attorney General.

Maintained by the state’s Attorney General.

Tuesday, 05. January 2021

reb00ted

Singaporan COVID-19 contact tracing data not private after all

The police now take the right to access the data for “authorized purposes”. With no clear process by which to limit those “authorized purposes” (ZDNet).

The police now take the right to access the data for “authorized purposes”. With no clear process by which to limit those “authorized purposes” (ZDNet).


MyDigitalFootprint

Peak Paradox

In the world full of human bias,  exploring paradox might help us explain our differences. Kaliya (Identity Woman) and I wrote “Humans want principles, society demands rules and businesses want to manage risk, can we reconcile the differences?  Have been noodling on that framework of linking purpose and rules with a few friends. This article explores how to look for a paradox within ou
In the world full of human bias,  exploring paradox might help us explain our differences. Kaliya (Identity Woman) and I wrote “Humans want principles, society demands rules and businesses want to manage risk, can we reconcile the differences?  Have been noodling on that framework of linking purpose and rules with a few friends. This article explores how to look for a paradox within our known cognitive bias and how to identify and manage differences.

----

I follow the excellent work of my friend Rory Sutherland and Nir Eyal (Nir & Far) who are both leading thinkers in behavioural science and economics along with the great Daniel Kahneman. I reflected following a call with the most read man on the planet Robbie Stamp CEO of BIOSS (a company who can assess capabilities for complex decision making), about how to frame bias and conflicts. We are aware that there are over 180 cognitive biases of human behaviour, which because of differences between us; create unique tensions, alignments, conflicts and paradoxes.

The 180 cognitive biases of human behaviour create unique tension, alignment, conflict and paradox.

The famous infographic below from Buster Benson on Medium https://medium.com/@buster has become foundational in presenting the array of human cognitive bias. Buster is well worth following, and I am looking forward to reading his new book "Why Are We Yelling? The Art of Productive Disagreement"

 

 

On another call with my friend, the insanely cleaver identity lawyer and polymath Scott David, we were noodling that we saw paradoxes within behavioural science, primarily when reflecting on the very long list of human biases.  I love his quote “we have to find the paradox, or we are probably in a model.” On a similar call with Viktor Mirovic  (CEO of KeenCorp - a company who identifies employee problems in real-time), we explored the gaps between our own bias and purpose and those biases/purpose that our companies have.  Viktor and I reflected on what happens when there is a delta between you and the company and the (a)effects on teams and individuals performance.  As you can imagine, this article and thinking are built ’the shoulders of giants and would not have come to be without those conversations and many others.  

The drawing below is to explore how to frame paradox within our known biases (maybe beliefs).  We will unpack the diagram, examine how we manage the differences, and recognise that mostly we don't; which leads to harmful stress. 

On discovering Peak Paradox

The four outer extremes are peaks in their own right; at these individual peaks, you can only hold one single view at the expense of all others.  Therefore there is no conflict at a peak, and no compromise as this is the only view that can be held.  

The axes are set up that on the X-axis (horizontal) is the conflict of our human purpose vs a commercial purpose. On the Y-axis (vertical) it is the conflict of individual vs everyone. 

Peak Individual Purpose.   At the exclusion of anything else, you are only interested in yourself.  Selflessness at the extreme.  You believe that you are sovereign (not having to ask anyone for permission or forgiveness), your voice and everything you say matters and everyone should agree. You are all-powerful and can do whatever you want and have the freedom and agency to do it.  

Peak Work Purpose.  Work in this context is commercial or economic. At the exclusion of anything else, the only reason a company exists is to deliver as much possible value to the shareholders.  Employees, customers, the environment does not matter in terms of exploitation.  The purpose is to be the biggest and most efficient beast on the planet and able to deliver enormous returns; Shareholder Primacy at its most pure. Simple, straightforward, no conflict and no compromise. Even to the point that rewarding the staff beyond fair would be a compromise.  Compliance is met with the minimal standard ensuring that nothing is wasted. 

Peak Society Purpose.  At the exclusion of anything else, we have to deliver and ensure there is no suffering and poverty for any living thing. Humans must have equal education, health and safety.  There must be total transparency and equality.  Everything is equally shared, and on-one has more power, agency or influence than anyone else.  

Peak Human Purpose.  At the exclusion of anything else, we are here to survive as long as possible and escape death, which we do by reproducing as much as we can with the broadest community we can.  We also have to adapt as fast as possible.  We have to meet our chemistry requirements to stay alive for as long as possible to adopt and reproduce at the expense of everything else.  Whilst all the peak purposes might be controversial (even to myself), saying purity of human purpose is chemistry/ biology might not go down very well. However, this is a model for framing thinking, so please go with it as it needs to be pure, and every other human purpose has conflicts with someone.

These extremes as peaks articulate that there is no conflict, no compromise, no tension - they are pure when at the extreme.  It is not that we have to agree with them but to recognise they can exist.    My gut says right-wing politics (given the interpretation of capitalist today and not its original meaning) follows the top edge between peak commercial purpose and peak individual purpose. Depending on the specific view individuals can be somewhere along the line, whereas parties are more complex in their overall position.   Left-wing political views (today's interruption more socialist) follow the bottom right edge between peak commercial purpose and peak society. Again individual views may hold the line, but parties are trying to find the right balance of majority votes, commercial activity, tax, redistribution and a fairer society.  Applying the same thinking Cults are likely to be positioned along the top left boundary between peak human purpose and peak individual purpose, whereas more fervent religious movements will tend towards the lower-left boundary between peak human purpose and peak society. Like political parties, world religions need a mass following for a voice and therefore are positioned with more paradoxes, which they solve with paradoxes.   

Peak Paradox.  The melting pot that is the middle of all of the peaks, the place where you are trying to rationalise all the extreme purposes into one acceptable position for everyone, but there is no resolution without compromise that will suit no-one.  Peak Paradox is likely to be unsustainable due to the conflicts and compromises required or may itself be a paradox in so much when there it feels like nothing is there, like the eye of the storm when there is complete calm.  It feels that many great thinkers and philosophers may try to find rest or stillness in this calm at peak paradox. There is a battle to get into the place of calm fighting the storms of opinions, and if you lose that moment of mindfulness, it is straight back into the storm. The unstable nature of standing on the point of a needle. This said:

Just because we may agree on the same peak purpose, that does not mean we can also agree on how to go about achieving or maintaining it.

Different peak purposes can have the same principles and values.  You come from different peaks towards a livable compromise; however, as individuals, you can have the same principles and values, making the acceptance of difference more acceptable. 

If there is no paradox or you cannot find one, you are at a boundary edge, where there is the greatest order, or at an extreme peak view. 

At peak paradox, there is the highest disorder, in terms of a variety of views.

It is evident that our long list of personality testing is to identify where you naturally identify with right now. You will change and adapt, but there is likely to be a natural affinity that tends towards one or more peaks.  

There are over 180 cognitive biases recognised,  from this diagram, we can unpack that you are unlikely to have them all at the same time, but a subset of them depending on where you locate yourself. 

What is the relation between bias and paradox?

When there are no biases, surely we have overcome all our objections, and we can deal with everyone’s unique and individual views, and we must be at “Peak Paradox.”

When there is no paradox, surely we are at an extreme view where there are no tensions, no conflicts, and we have no biases that can distract us from accepting every position equally.

Most of us do not see the paradox in front of us. Still, we have many biases, suggesting that Peak Bias (number of biases), which means we reject most ideas accepting only a few, will occur early in a distribution. 

As we can deal with more complex compromises, we can see more paradoxes in any position but can cope with the conflicts and tensions that arise, suggesting a long tail to peak paradox. 

4 case studies

I wrote about how to uncover your purpose here, but I can now see that personal purpose maps differently to paradox and bias. Mapping a few personal or individual purposes. 

Invent and Crete.  “Necessity is the mother of all invention” and “Creativity is intelligence having fun” sum up rather well that part of our human make-up is to find ways to create or invent something that solves a problem we see or understand.  Our personal purpose is to invent and create as we see what everyone else sees but can think what no-one else has thought. Irrespective of such a personal goal it maps all of the blue area on the Peak Paradox diagram as every paradox creates a problem that needs to be solved.  Being creative as a purpose does not mean you will find anyone else on the same journey.

Creating Impact.  Some individuals desire to have more of an impact whilst alive and some, because of what they do, have a lasting effect that changes our human journey, some good and some not so.  This is the point about creating impact as a purpose; it is like all items that humans make: they can be used as tools for good or weapons for bad.  Creating impact creates a paradox of how to create something that cannot be used for harm, but often we create something without seeing the harm.  Our individual purpose is itself in conflict with the outcome we want.   At every point under the blue area on the Peak Paradox chat, creating impact for good or ill is possible at every point. 

Better ancestors.  What changes when you consider governance for the next 1000 years is a post about how we have to become better stewards and ancestors.  We have to be more considerate if we want others to walk our path and not burn the earth.  As a thinking piece, I focussed on the paradox of high commercial pressure, being between citizens, giving our children life and enabling more individuals to have agency.  Almost peak paradox. Perhaps this is the case with ESG as a conceptual issue, we have to compromise, and we are not in agreement on what we need to compromise on?

Perhaps this is the case with ESG as a conceptual issue, we have to compromise, and we are not in agreement on what we need to compromise on?

Being Useful.  There appears to be a human need that presents as wanting to be loved, wanted or useful.  Over time we go through these different phases on a cyclic basis throughout life. What we conceive as “being useful” is profoundly personal and therefore could be anywhere in the central blue area on the Peak Paradox drawing. Being Useful today can mean you want to do well in assisting, where you are employed, achieve its goals or mission, and help you achieve your personal goal of more agency.  Equally, being useful can mean creating less poverty and suffering in the world.  Anyone who describes themselves with a desire to be useful is unlikely to find a massive community who have the same idea of what that means.

Personal purposes are both aligned to all peaks, and also in conflict with all the peaks, it is why we should see the paradoxes in our own life's purpose and find we cannot rest on one.  We have to settle for a compromise that we find acceptable and does not bring conflict, tension or anxiety. 

Towards better performance, strategy and judgement. 

Teams and Culture. We have one culture; this is how we do it.  These are our rules; these are our processes.  We have one team, and all our thinking is aligned.  After reading this, you might be as sceptical about such thoughts as I am.  In my article “Humans want principles, society demands rules and businesses want to manage risk, can we reconcile the differences?   I did not connect the dots between the delta or gap between where the company has decided to operate with its paradox, compromises, tensions and conflicts and how they would align to the individuals inside the organisation.  I am sure there is a link here to transformation issues as well.    KeenCorp is looking at measuring this, something I am going to watch 

Philanthropy is a private initiative from individuals. Interesting Wikipedia defines philanthropy as for the public good, focusing on the quality of life. However, often we see that the person who is giving has an idea of public good and priority that is not aligned to mine, which creates friction.   Philanthropy has an interesting dynamic about how much it is for PR and how much it is indeed the individual’s purpose.  Might have to analyse the Bill and Melinda Gates Foundation and see what it says about what paradoxes they are prepared to live with and help understand the criticism and support offered. 

Investors, directors and the company.  As a group, you can imagine that they are one the closest to a peak purpose.  Investors as equity investors believe, one would hope, in the purity of shareholder primacy and would probably be outright supporters of this single focus.  However, since we have lived that view, we are now much more paradoxical.  ESG and stewardship codes mean that the purity has become a complex mix of different personal compromises, which currently are not disclosed and may be in conflict to the culture or purpose of the company that they invest into.  The relationship between investors and Directors is also changing. However, it appears that the capital providers are willing to embrace ESG and a watered-down version of peak Shareholder Primacy. However, Directos, KPI, remuneration committees and self-interested processes might be creating a level of friction to nobel ESG change that we did not anticipate. Organisational wellness should also now be a new measure and reported on, but how? 

Governance.  Governance of old did not have to embrace compromise, conflict or tensions.  Investors and founders understood the simplicity of a very pure set of drivers.  The founding team had largely sorted out the paradox and how they could work together.  Indeed anyone who could not be soon off.  Governance was then ensuring the right NorthStar, the suitable vessel and the right team.  This model is no longer good enough. Suddenly, the directors are having to face up to conflicting peak purposes pulling in different directions by directors who have their own views and teams who also have a  significant voice.  Added to this is the dependence on the ecosystem who themselves will reach a different compromise which asks fundamental questions about your interaction with them.     


Take away

If there is no paradox, compromise or conflict, you are in a model pretending it is life.

If it is too good to be true, you are probably at an extreme as there is no tension, conflict or paradox. 

Are we in a place where our compromises are good for us? Can we perform?

We can identify stress that is probably harmful as we have compromised to someone else’s force, dominance, position and have ended up in a place where we do not naturally belong.

Is there an alignment, understanding and communication between our compromises and those around us in our team? Can we see and talk about the paradoxes in the team's decisions?

Neuro, race and gender diversity are critical for success, understanding our natural positions and compromises makes sense. Knowing the delta between our team is crucial.

Being able to talk to and understand all positions means we will be in a less divided and divisional world.


Next questions for me

Does this model allow me to plot products and companies to determine alignment? 

Ethics; is it our ability to deal with compromise and conflict and find a position we find acceptable?


@tonyfish  Jan 2021



reb00ted

Roaring 20s?

Seeing another prediction. I’m having some difficulties envisioning this for anybody than the, say, top-10% of the population in countries such as the US. Family balance sheets are wiped out across the board. How can you roar?

Seeing another prediction. I’m having some difficulties envisioning this for anybody than the, say, top-10% of the population in countries such as the US. Family balance sheets are wiped out across the board. How can you roar?


MyDigitalFootprint

Ethical Fading and Moral Disengagement

Brillent video explaining Ethical Fading and Moral Disengagement Source: https://ethicsunwrapped.utexas.edu/video/ethical-fading

Brillent video explaining Ethical Fading and Moral Disengagement
Source: https://ethicsunwrapped.utexas.edu/video/ethical-fading

Monday, 04. January 2021

Tim Bouma's Blog

The Digital Identity Standards To Bet On In 2021

Photo by Edge2Edge Media on Unsplash Author’s note: This is the sole opinion of the author and may be revised at any time. The views and positions expressed do not necessarily reflect that of the author’s employer nor any involved organizations, committees, or working groups. If someone were to ask me: “What are the standards you are betting on for 2021?”, this would be my answer:
Photo by Edge2Edge Media on Unsplash

Author’s note: This is the sole opinion of the author and may be revised at any time. The views and positions expressed do not necessarily reflect that of the author’s employer nor any involved organizations, committees, or working groups.

If someone were to ask me: “What are the standards you are betting on for 2021?”, this would be my answer:

There are hundreds of ‘digital identity’ standards out there. I have winnowed down the list to three — two technical standards and one non-technical standard:

W3C Decentralized Identifiers (DIDs) v1.0 for a new type of identifier that enables verifiable, decentralized digital identity. A DID identifies any subject (e.g., a person, organization, thing, data model, abstract entity, etc.) that the controller of the DID decides that it identifies. W3C Verifiable Credentials Data Model 1.0 a standardized specification that provides a mechanism to express credentials on the Web in a way that is cryptographically secure, privacy-respecting, and machine-verifiable. CAN/CIOSC 103–1:2020 Digital Trust And Identity — Part 1 specifies minimum requirements and a set of controls for creating and maintaining trust in digital systems and services that, as part of an organization’s mandate, assert and or consume identity and credentials in data pertaining to people and Organizations.

Admittedly, I am writing this for the Canadian context (as the third choice is Canadian-only, so insert your own national or international standard here), but the main reasons I have chosen these three is because they represent a new way forward to develop a digital ecosystem that is open, inclusive, and balanced in favour towards the individual.

I realize that there are many more standards at play, but it is my belief that it is these three that will enable trusted digital identity across many ecosystems — across industries and across political boundaries.

That’s my start for 2021!

Sunday, 03. January 2021

Jon Udell

Why public phones still exist

My superpower has always been finding new uses for old tech. In the late 90s I dusted off the venerable NNTP server, which had been the backbone of the Usenet, and turned it into my team’s Slack. In the late 2000s I used iCalendar to make citywide event calendars. In the late 2010s I went … Continue reading Why public phones still exist

My superpower has always been finding new uses for old tech. In the late 90s I dusted off the venerable NNTP server, which had been the backbone of the Usenet, and turned it into my team’s Slack. In the late 2000s I used iCalendar to make citywide event calendars. In the late 2010s I went deep into SQL.

It’s always intensely pragmatic. But also, I can’t deny, whimsical.

In that spirit, I offer you the public pay phone at the Pinnacles Visitor Center. I stayed in that campground on a road trip just before the election. Given the tense political and epidemiological situation, I’d promised to message home regularly. There was no cell service in the park so I headed over to the office. It was closed, so I sat on the bench and connected to their WiFi. Or tried to. You could connect, sometimes, but you couldn’t move any data. The router was clearly in need of a reboot.

The only option left was the public phone. I can’t remember the last time I used one. Most people alive today have, perhaps, never used one. But there it was, so I gave it a shot.

Once upon a time, you could pick up the handset, dial 0 for operator, and place a so-called collect (charge-reversed) call. Now dialing 0 gets you nowhere.

The instructions taped to the phone (in the 90s I’m guessing) say you can call an 800 number, or use a calling card. I remember calling cards, I had one once. Not a thing lately.

And then there was this: “Dial 611 for help.”

Me: 611

611: Hello, this is Steve.

Me: I’m at the Pinnacles Visitor Center trying to send a message.

Steve: Use the WiFi.

Me: I can’t, it’s broken.

Steve: Huh, that’s interesting. Let me see if I can reboot the router.

And he did. So there you have it. The public phone still provides a valuable service. Its mission has evolved over the years. Nowadays, it exists to summon Steve the IT guy who can fix the WiFi by turning it off and on again.

Works like a charm!

Friday, 01. January 2021

Mike Jones: self-issued

Near-Final Second W3C WebAuthn and FIDO2 CTAP Specifications

The W3C WebAuthn and FIDO2 working groups have been busy this year preparing to finish second versions of the W3C Web Authentication (WebAuthn) and FIDO2 Client to Authenticator Protocol (CTAP) specifications. While remaining compatible with the original standards, these second versions add additional features, among them for user verification enhancements, manageability, enterprise features, and a

The W3C WebAuthn and FIDO2 working groups have been busy this year preparing to finish second versions of the W3C Web Authentication (WebAuthn) and FIDO2 Client to Authenticator Protocol (CTAP) specifications. While remaining compatible with the original standards, these second versions add additional features, among them for user verification enhancements, manageability, enterprise features, and an Apple attestation format. Near-final review drafts of both have been published:

Web Authentication: An API for accessing Public Key Credentials, Level 2, W3C Candidate Recommendation Snapshot, 22 December 2020 Client to Authenticator Protocol (CTAP), Review Draft, December 08, 2020

Expect these to become approved standards in early 2021. Happy New Year!


SecEvent Delivery specs are now RFCs 8935 and 8936

The SecEvent Delivery specifications, “Push-Based Security Event Token (SET) Delivery Using HTTP” and “Poll-Based Security Event Token (SET) Delivery Using HTTP”, are now RFC 8935 and RFC 8936. Both deliver Security Event Tokens (SETs), which are defined by RFC 8417. The abstracts of the specifications are: Push-Based Security Event Token (SET) Delivery Using HTTP: This […]

The SecEvent Delivery specifications, “Push-Based Security Event Token (SET) Delivery Using HTTP” and “Poll-Based Security Event Token (SET) Delivery Using HTTP”, are now RFC 8935 and RFC 8936. Both deliver Security Event Tokens (SETs), which are defined by RFC 8417. The abstracts of the specifications are:

Push-Based Security Event Token (SET) Delivery Using HTTP:

This specification defines how a Security Event Token (SET) can be delivered to an intended recipient using HTTP POST over TLS. The SET is transmitted in the body of an HTTP POST request to an endpoint operated by the recipient, and the recipient indicates successful or failed transmission via the HTTP response.

Poll-Based Security Event Token (SET) Delivery Using HTTP:

This specification defines how a series of Security Event Tokens (SETs) can be delivered to an intended recipient using HTTP POST over TLS initiated as a poll by the recipient. The specification also defines how delivery can be assured, subject to the SET Recipient’s need for assurance.

These were designed with use cases such as Risk & Incident Sharing and Collaboration (RISC) and Continuous Access Evaluation Protocol (CAEP) in mind, both of which are happening in the OpenID Shared Signals and Events Working Group.

Monday, 28. December 2020

Phil Windley's Technometria

The Generative Self-Sovereign Internet

Summary: The self-sovereign internet, a secure overlay on the internet, provides the same capacity to produce change by numerous, unaffiliated and uncoordinated actors as the internet itself. The generative nature of the self-sovereign internet is underpinned by the same kind of properties that make the internet what it is, promising a more secure and private, albeit no less useful, internet for t

Summary: The self-sovereign internet, a secure overlay on the internet, provides the same capacity to produce change by numerous, unaffiliated and uncoordinated actors as the internet itself. The generative nature of the self-sovereign internet is underpinned by the same kind of properties that make the internet what it is, promising a more secure and private, albeit no less useful, internet for tomorrow.

This is part one of a two part series on the generativity of SSI technologies. This article explores the properties of the self-sovereign internet and makes the case that they justify its generativity claims. The second part will explore the generativity of verifiable credential exchange, the essence of self-sovereign identity.

In 2005, Jonathan Zitrain wrote a compelling and prescient examination of the generative capacity of the Internet and its tens of millions of attached PCs. Zittrain defined generativity thus:

Generativity denotes a technology’s overall capacity to produce unprompted change driven by large, varied, and uncoordinated audiences.

Zittrain masterfully describes the extreme generativity of the internet and its attached PCs, explains why openness of both the network and the attached computers is so important, discusses threats to the generativity nature of the internet, and proposes ways that the internet can remain generative while addressing some of those threats. While the purpose of this article is not to review Zittrain's paper in detail, I recommend you take some time to explore it.

Generative systems use a few basic rules, structures, or features to yield behaviors that can be extremely varied and unpredictable. Zittrain goes on to lay out the criteria for evaluating the generativity of a technology:

Generativity is a function of a technology’s capacity for leverage across a range of tasks, adaptability to a range of different tasks, ease of mastery, and accessibility.

This sentence sets forth four important criteria for generativity:

Capacity for Leverage—generative technology makes difficult jobs easier—sometimes possible. Leverage is measured by the capacity of a device to reduce effort. Adaptability—generative technology can be applied to a wide variety of uses with little or no modification. Where leverage speaks to a technology's depth, adaptability speaks to its breadth. Many very useful devices (e.g. airplanes, saws, and pencils) are nevertheless fairly narrow in their scope and application. Ease of Mastery—generative technology is easy to adopt and adapt to new uses. Many billions of people use a PC (or mobile device) to perform tasks important to them without significant skill. As they become more proficient in its use, they can apply it to even more tasks. Accessibility—generative technology is easy to come by and access. Access is a function of cost, deployment, regulation, monopoly power, secrecy, and anything else which introduces artificial scarcity.

The identity metasystem I've written about in the past is composed of several layers that provide its unique functionality. This article uses Zittrain's framework, outlined above, to explore the generativity of what I've called the Self-Sovereign Internet, the second layer in the stack shown in Firgure 1. A future article will discuss the generativity of credential exchange at layer three.

Figure 1: SSI Stack (click to enlarge) The Self-Sovereign Internet

In DIDComm and the Self-Sovereign Internet, I make the case that the network of relationships created by the exchange of decentralized identifiers (layer 2 in Figure 1) forms a new, more secure layer on the internet. Moreover, the protocological properties of DIDComm make that layer especially useful and flexible, mirroring the internet itself.

This kind of "layer" is called an overlay network. An overlay network comprises virtual links that correspond to a path in the underlying network. Secure overlay networks rely on an identity layer based on asymmetric key cryptography to ensure message integrity, non-repudiation, and confidentiality. TLS (HTTPS) is a secure overlay, but it is incomplete because it's not symmetrical. Furthermore, it's relatively inflexible because it overlays a network layer using a client-server protocol1.

In Key Event Receipt Infrastructure (KERI) Design, Sam Smith makes the following important point about secure overlay networks:

The important essential feature of an identity system security overlay is that it binds together controllers, identifiers, and key-pairs. A sender controller is exclusively bound to the public key of a (public, private) key-pair. The public key is exclusively bound to the unique identifier. The sender controller is also exclusively bound to the unique identifier. The strength of such an identity system based security overlay is derived from the security supporting these bindings. From Key Event Receipt Infrastructure (KERI) Design
Referenced 2020-12-21T11:08:57-0700

Figure 2 shows the bindings between these three components of the secure overlay.

Figure 2: Binding of controller, authentication factors, and identifiers that provide the basis for a secure overlay network. (click to enlarge)

In The Architecture of Identity Systems, I discuss the strength of these critical bindings in various identity system architectures. The key point for this discussion is that the peer-to-peer network created by peer DID exchanges constitute an overlay with an autonomic architecture, providing not only the strongest possible bindings between the controller, identifiers, and authentication factors (public key), but also not needing an external trust basis (like a ledger) because they are self-certifying.

DIDs allow us to create cryptographic relationships, solving significant key management problems that have plagued asymmetric cryptography since it's inception. Consequently, regular people can use a general purpose secure overlay network based on DIDs. The DID network that is created when people use these relationships provides a protocol, DIDComm, that is every bit as flexible and useful as is TCP/IP.

Consequently, communications over a DIDComm-enabled peer-to-peer network are as generative as the internet itself. Thus, the secure overlay network formed by DIDComm connections represents a self-sovereign internet, emulating the underlying internet's peer-to-peer messaging in a way that is both secure and trustworthy2 without the need for external third parties3.

Properties of the Self-Sovereign Internet

In World of Ends, Doc Searls and Dave Weinberger enumerate the internet's three virtues:

No one owns it. Everyone can use it. Anyone can improve it.

These virtues apply to the self-sovereign internet as well. As a result, the self-sovereign internet displays important properties that support it's generativity. Here are the most important:

Decentralized—decentralization follows directly from the fact that no one owns it. This is the primary criterion for judging the degree of decentralization in a system.

Heterarchical—a heterarchy is a "system of organization where the elements of the organization are unranked (non-hierarchical) or where they possess the potential to be ranked a number of different ways." Nodes in a DIDComm-based network relate to each other as peers. This is a heterarchy; there is no inherent ranking of nodes in the architecture of the system.

Interoperable—regardless of what providers or systems we use to connect to the self-sovereign internet, we can interact with any other principles who are using it so long as they follow protocol4.

Substitutable—The DIDComm protocol defines how systems that use it must behave to achieve interoperability. That means that anyone who understands the protocol can write software that uses DIDComm. Interoperability ensure that we can operate using a choice of software, hardware, and services without fear of being locked into a proprietary choice. Usable substitutes provide choice and freedom.

Reliable and Censorship Resistant—people, businesses, and others must be able to use the secure overlay network without worrying that it will go down, stop working, go up in price, or get taken over by someone who would do it and those who use it harm. This is larger than mere technical trust that a system will be available and extends to the issue of censorship.

Non-proprietary and Open—no one has the power to change the self-sovereign internet by fiat. Furthermore, it can't go out of business and stop operation because its maintenance and operation are distributed instead of being centralized in the hands of a single organization. Because the self-sovereign internet is an agreement rather than a technology or system, it will continue to work.

The Generativity of the Self-Sovereign Internet

Applying Zittrain's framework for evaluating generativity is instructive for understanding the generative properties of the self-sovereign internet.

Capacity for Leverage

In Zittrain's words, leverage is the extent to which an object "enables valuable accomplishments that otherwise would be either impossible or not worth the effort to achieve." Leverage multiplies effort, reducing the time and cost necessary to innovate new capabilities and features. Like the internet, DIDComm's extensibility through protocols enables the creation of special-purpose networks and data distribution services on top of it. By providing a secure, stable, trustworthy platform for these services, DIDComm-based networks reduce the effort and cost associated with these innovations.

Like a modern operating system's application programming interface (API), DIDComm provides a standardized platform supporting message integrity, non-repudiation, and confidentiality. Programmers get the benefits of a trusted message system without need for expensive and difficult development.

Adaptability

Adaptability can refer to a technology's ability to be used for multiple activities without change as well as its capacity for modification in service of new use cases. Adaptability is orthogonal to capacity for leverage. An airplane, for example, offers incredible leverage, allowing goods and people to be transported over long distances quickly. But airplanes are neither useful in activities outside transportation or easily modified for different uses. A technology that supports hundreds of use cases is more generative than one that is useful in only a few.

Like TCP/IP, DIDComm makes few assumptions about how the secure messaging layer will be used. Thus the network formed by the nodes in a DIDComm network can be adapted to any number of applications. Moreover, because a DIDComm-based network is decentralized and self-certifying, it is inherently scalable for many uses.

Ease of Mastery

Ease of use refers to the ability of a technology to be easily and broadly adapted and adopted. The secure, trustworthy platform of the self-sovereign internet allows developers to create applications without worrying about the intricacies of the underlying cryptography or key management.

At the same time, because of its standard interface and protocol, DIDComm-based networks can present users with a consistent user experience that reduces the skill needed to establish and use connections. Just like a browser presents a consistent user experience on the web, a DIDComm agent can present users with a consistent user experience for basic messaging, as well as specialized operations that run over the basic messaging system.

Of special note is key management, which has been the Achilles heal of previous attempts at secure overlay networks for the internet. Because of the nature of decentralized identifiers, identifiers are separated from the public key, allowing the keys to be rotated when needed without also needing to refresh the identifier. This greatly reduces the need for people to manage or even see keys. People focus on the relationships and the underlying software manages the keys.5

Accessibility

Accessible technologies are easy to acquire, inexpensive, and resistant to censorship. DIDComm's accessibility is a product of its decentralized and self-certifying nature. Protocols and implementing software are freely available to anyone without intellectual property encumbrances. Multiple vendors, and even open-source tools can easily use DIDComm. No central gatekeeper or any other third party is necessary to initiate a DIDComm connection in service of a digital relationship. Moreover, because no specific third parties are necessary, censorship of use is difficult.

Conclusion

Generativity provides decentralized actors to create cooperating, complex structures and behavior. No one person or group can or will think of all the possible uses, but each is free to adapt the system to their own use. The architecture of the self-sovereign internet exhibits a number of important properties. The generativity of the self-sovereign internet depends on those properties. The true value of the self-sovereign internet is that it provides an leveragable, adaptable, usable, accessible, and stable platform upon which others can innovate.

Notes Implementing general-purpose messaging on HTTP is not straightforward, especially when combined with non-routable IP addresses for many clients. On the other hand, simulating client-server interactions on a general-purpose messaging protocol is easy. I'm using "trust" in the cryptographic sense, not in the reputational sense. Cryptography allows us to trust the fidelity of the communication but not its content. Admittedly, the secure overlay is running on top of a network with a number of third parties, some benign and others not. Part of the challenge of engineering a functional secure overlay with self-sovereignty it mitigating the effects that these third parties can have within the self-sovereign internet. Interoperability is, of course, more complicated than merely following the protocols. Daniel Hardman does an excellent job of discussing this for verifiable credentials (a protocol that runs over DIDComm), in Getting to Practical Interop With Verifiable Credentials. More details about some of the ways software can greatly reduce the burden of key management when things go wrong can be found in What If I Lose My Phone? by Daniel Hardman.

Photo Credit: Seed Germination from USDA (CC0)

Tags: generative internet identity ssi didcomm decentralized+identifiers self-sovereign+internet

Sunday, 27. December 2020

Doc Searls Weblog

We’ve seen this movie before

When some big outfit with a vested interest in violating your privacy says they are only trying to save small business, grab your wallet. Because the game they’re playing is misdirection away from what they really want. The most recent case in point is Facebook, which ironically holds the world’s largest database on individual human […]

When some big outfit with a vested interest in violating your privacy says they are only trying to save small business, grab your wallet. Because the game they’re playing is misdirection away from what they really want.

The most recent case in point is Facebook, which ironically holds the world’s largest database on individual human interests while also failing to understand jack shit about personal boundaries.

This became clear when Facebook placed the ad above and others like it in major publications recently, and mostly made bad news for itself. We saw the same kind of thing in early 2014, when the IAB ran a similar campaign against Mozilla, using ads like this:

That one was to oppose Mozilla’s decision to turn on Do Not Track by default in its Firefox browser. Never mind that Do Not Track was never more than a polite request for websites to not be infected with a beacon, like those worn by marked animals, so one can be tracked away from the website. Had the advertising industry and its dependents in publishing simply listened to that signal, and respected it, we might never have had the GDPR or the CCPA, both of which are still failing at the same mission. (But, credit where due: the GDPR and the CCPA have at least forced websites to put up insincere and misleading opt-out popovers in front of every website whose lawyers are scared of violating the letter—but never the spirit—of those and other privacy laws.)

The IAB succeeded in its campaign against Mozilla and Do Not Track; but the the victory was Pyrrhic, because users decided to install ad blockers instead, which by 2015 was the largest boycott in human history. Plus a raft of privacy laws, with more in the pipeline.

We also got Apple on our side. That’s good, but not good enough.

What we need are working tools of our own. Examples: Global Privacy Control (and all the browsers and add-ons mentioned there), Customer Commons#NoStalking term, the IEEE’s P7012 – Standard for Machine Readable Personal Privacy Terms, and other approaches to solving business problems from the our side—rather than always from the corporate one.

In those movies, we’ll win.

Because if only Apple wins, we still lose.

Dammit, it’s still about what The Cluetrain Manifesto said in the first place, in this “one clue” published almost 21 years ago:

we are not seats or eyeballs or end users or consumers.
we are human beings — and out reach exceeds your grasp.
deal with it.

We have to make them deal. All of them. Not just Apple. We need code, protocols and standards, and not just regulations.

All the projects linked to above can use some help, plus others I’ll list here too if you write to me with them. (Comments here only work for Harvard email addresses, alas. I’m doc at searls dot com.)

Saturday, 26. December 2020

Doc Searls Weblog

Wonder What?

Our Christmas evening of cinematic indulgence was watching Wonder Woman 1984, about which I just posted this, elsewhere on the Interwebs: I mean, okay, all “super” and “enhanced” hero (and villain) archetypes are impossible. Not found in nature. You grant that. After a few thousand episodes in the various franchises, one’s disbelief becomes fully suspended. So […]

Our Christmas evening of cinematic indulgence was watching Wonder Woman 1984, about which I just posted this, elsewhere on the Interwebs:

I mean, okay, all “super” and “enhanced” hero (and villain) archetypes are impossible. Not found in nature. You grant that. After a few thousand episodes in the various franchises, one’s disbelief becomes fully suspended. So when you’ve got an all-female island of Amazons (which reproduce how?… by parthenogenesis?) playing an arch-Freudian Greco-Roman Quidditch, you say hey, why not? We’re establishing character here. Or backstory. Or something. You can hang with it, long as there are a few connections to what might be a plausible reality, and while things move forward in a sensible enough way. And some predictability counts. For example, you know the young girl, this movie’s (also virgin-birthed) Anakin Skywalker, is sure to lose the all but endless Quidditch match, and will learn in losing a lesson (taught by … who is that? Robin Wright? Let’s check on one of our phones) that will brace the front end of what turns out at the end of the story to be its apparent moral arc.

And then, after the girl grows up to be an introverted scientist-supermodel who hasn’t aged since WWI (an item that hasn’t raised questions with HR since long before it was called “Personnel,” and we later learn has been celibate or something ever since her only-ever boyfriend died sixty-four years earlier while martyring his ass in a plane crash you’re trying to remember from the first movie) has suddenly decided, after all this time, to start fighting crime with her magic lasso and her ability to leap shopping mall atria in a single bound; and then, after same boyfriend inexplicably comes back from the dead to body-snatch some innocent dude, they go back to hugging and smooching and holding hands like the intervening years of longing (her) and void (him) were no big deals, and then they jack an idle (and hopefully gassed up) F111, which in reality doesn’t have pilot-copilot seats side-by-side (or even a co-pilot, beging a single-seat plane), and which absolutely requires noise-isolating earphones this couple doesn’t have, because afterburner noise in the cockpit in one of those mothers is about 2000db, and the undead boyfriend, who flew a Fokker or something in the prior movie, knows exactly and how to fly a jet that only the topmost of guns are allowed to even fantasize about, and then he and Wondermodel have a long conversation on a short runway during which they’re being chased by cops, and she kinda doubts that one of the gods in her polytheistic religion have given her full powers to make a whole plane invisible to radar, which she has to explain to her undead dude in 1984 (because he wouldn’t know about that, even though he knows everything else about the plane), and the last thing she actually made disappear was a paper cup, and then they somehow have a romantic flight, without refueling, from D.C. to a dirt road in an orchard somewhere near Cairo, while in the meantime the most annoying and charmless human being in human history—a supervillain-for-now whose one human power was selling self-improvement on TV—causes a giant wall to appear in the middle of a crowded city while apparently not killing anyone… Wholly shit.

And what I just described was about three minutes in the midst of this thing.

But we hung with it, in part because we were half-motivated to see if it was possible to tally both the impossibilities and plot inconsistencies of the damn thing. By the time it ended, we wondered if it ever would.

Bonus link.

Thursday, 24. December 2020

Doc Searls Weblog

A simple suggestion for Guilford College

Guilford College made me a pacifist. This wasn’t hard, under the circumstances. My four years there were the last of the 1960s, a stretch when the Vietnam War was already bad and getting much worse. Nonviolence was also a guiding principle of the civil rights movement, which was very active and local at the time, and […]

Guilford College made me a pacifist.

This wasn’t hard, under the circumstances. My four years there were the last of the 1960s, a stretch when the Vietnam War was already bad and getting much worse. Nonviolence was also a guiding principle of the civil rights movement, which was very active and local at the time, and pulled me in as well. I was also eligible for the draft if I dropped out. Risk of death has a way of focusing one’s mind.

As a Quaker college, this was also Guilford’s job. Hats off: I learned a lot, and enjoyed every second of it.

These days, however, Guilford—like lots of other colleges and universities—is in trouble. Scott Galloway and his research team at NYU do a good job of sorting out every U.S. college’s troubles here:

You’ll find Guilford in the “struggle” quadrant, top left. That one contains “Tier-2 schools with one or more comorbidities, such as high admit rates (anemic waiting lists), high tuition, or scant endowments.”

So I’d like to help Guilford, but not (yet) with the money they constantly ask me for. Instead, I have some some simple advice: teach peace. Become the pacifist college. There’s a need for that, and the position is open. A zillion other small liberal arts colleges do what Guilford does. Replace “Guilford” on the page at that link with the name of any other good small liberal arts college and it’ll work for all of them. But none of the others teach peace, or wrap the rest of their curricular offerings around that simple and straightforward purpose. Or are in a position to do that. Guilford is.

Look at it this way: any institution can change in a zillion different ways; but the one thing it can’t change is where it comes from. Staying true to that is one of the strongest, most high-integrity things a college can do. By positioning around peace and pacifism, Guilford will align with its origins and stand alone in a field that will inevitably grow—and must for our species is to survive and thrive in an overcrowded and rapidly changing world.

Yes, there are a bunch of Quaker colleges, and colleges started by Quakers. (Twenty by this count). And they include some names bigger than Guilford’s: Cornell, Bryn Mawr, Haverford, Johns Hopkins. But none are positioned to lead on peace and pacifism, and only a few could be.(Earlham for sure. Maybe Wilmington.) The position is open, and  Guilford should take it.

Fortuitously, a few days ago I got an email from Ed Winslow, chair of Guilford’s Board of Trustees, that begins with this paragraph:

The Board of Trustees met on Dec. 15 to consider the significant feedback we have received and for a time of discernment. In that spirit, we have asked President Moore to pause implementation of the program prioritization while the Board continues to listen and gather input from those of you who wish to offer it. We are hearing particularly from alumni who are offering fundraising ideas. We are also hearing internally and from those in the wider education community who are offering ideas as well.

So that’s my input: own the Peace Position.

For fundraising I suggest an approach I understand is implemented by a few other institutions (I’m told Kent State is one): tell alumni you’re done asking for money constantly and instead ask only to be included in their wills. I know this is contrary to most fundraising advice; but I believe it will work—and does, for some schools. Think about it: just knowing emails from one’s alma mater aren’t almost always shakedowns for cash is a giant benefit by itself.

In case anyone at Guilford wonders who the hell I am and why my advice ought to carry some weight, forgive me while I waive modesty and present these two facts:

On the notable Guilford alumni list, I’m tops in search results. I even beat Howard Coble, Tom Zachary, M.L. Carr, Bob Kauffman and World B. Free. I was a success in the marketing business (much of it doing positioning) for several decades of my professional life.

So there ya go.

Peace, y’all.

Wednesday, 23. December 2020

Bill Wendel's Real Estate Cafe

#UncoupleREFees: Will third price-fixing lawsuit unleash consumer savings in Mass?

BREAKING NEWS: A third massive lawsuit has been filed to #UnCoupleREFees and this time it’s focused on the MLS (Multiple Listing Service) in Massachusetts plus… The post #UncoupleREFees: Will third price-fixing lawsuit unleash consumer savings in Mass? first appeared on Real Estate Cafe.

BREAKING NEWS: A third massive lawsuit has been filed to #UnCoupleREFees and this time it’s focused on the MLS (Multiple Listing Service) in Massachusetts plus…

The post #UncoupleREFees: Will third price-fixing lawsuit unleash consumer savings in Mass? first appeared on Real Estate Cafe.


Identity Woman

Human Centered Security Podcast

I was invited to join Heidi Trost to join her on my new podcast focused on Human Centered Security. We had a great chat focused on Self-Sovereign Identity. You can find it here on the Web, Spotifiy or Apple Podcast In this episode we talk about: What Kaliya describes as a new “layer” to the […] The post Human Centered Security Podcast appeared first on Identity Woman.

I was invited to join Heidi Trost to join her on my new podcast focused on Human Centered Security. We had a great chat focused on Self-Sovereign Identity. You can find it here on the Web, Spotifiy or Apple Podcast In this episode we talk about: What Kaliya describes as a new “layer” to the […]

The post Human Centered Security Podcast appeared first on Identity Woman.


Nader Helmy

Intro to MATTR Learn Concepts

In the world of decentralized identity and digital trust, there are a variety of new concepts and topics that are frequently referenced, written, and talked about, but rarely is there a chance to introduce these concepts formally to audiences who aren’t already familiar with them. For this reason, we have created a new “Learn Concepts” series to outline the the fundamental building blocks ne

In the world of decentralized identity and digital trust, there are a variety of new concepts and topics that are frequently referenced, written, and talked about, but rarely is there a chance to introduce these concepts formally to audiences who aren’t already familiar with them.

For this reason, we have created a new “Learn Concepts” series to outline the the fundamental building blocks needed to understand this new technology paradigm and explore the ways that MATTR thinks about and understands the critical issues in the space.

Over on our MATTR Learn site, we have been building out a variety of resources to assist developers and architects with understanding the MATTR universe of tools and products. We are happy to announce we have updated the site to include this new educational content series alongside our existing resources.

Our Learn Concepts series covers the following topics:

Web of Trust 101 Digital Wallets Verifiable Data Semantic Web Selective Disclosure Trust Frameworks

To facilitate context sharing, each of these Learn Concepts has a distinct Medium post with a permanent URL in addition to being published on our MATTR Learn site. We will keep these resources up to date to make sure they remain evergreen and relevant to newcomers in the space.

We are excited to share what we’ve learned on our journey, and we look forward to adapting and expanding this knowledge base as standards progress and technologies mature.

Intro to MATTR Learn Concepts was originally published in MATTR on Medium, where people are continuing the conversation by highlighting and responding to this story.


Learn Concepts: Trust Frameworks

Trust frameworks are a foundational component of the web of trust. A trust framework is a common set of best practice standards-based rules that ensure minimum requirements are met for security, privacy, identification management and interoperability through accreditation and governance. These operating rules provide a common framework for ecosystem participants, increasing trust between them

Trust frameworks are a foundational component of the web of trust. A trust framework is a common set of best practice standards-based rules that ensure minimum requirements are met for security, privacy, identification management and interoperability through accreditation and governance. These operating rules provide a common framework for ecosystem participants, increasing trust between them.

As digital service delivery models mature, it is essential that information is protected as it travels across jurisdictional and organizational boundaries. Trust frameworks define and bring together the otherwise disparate set of best practice principles, processes, standards that apply when it comes to collecting and sharing information on the web. As individuals and entities increasingly share their information cross contextually, across industry boundaries, trust frameworks provide the common set of rules that apply regardless of such differences. For example, service providers ranging from government agencies, banks and telecommunication companies, to health care providers could all follow the same set of data sharing practices under one trust framework. This macro application serves to reduce the need for bilateral agreements and fragmentation across industry. Ultimately trust frameworks serve to increase trust, improve efficiencies, and deliver significant economic and social benefits.

Some use-cases will require more detailed rules to be established than those set out in a trust framework with broad scope. Where this is the case, more detailed rules around specific hierarchies and roles can be established within the context of the higher order trust framework. The goal is always for the components of the framework to be transparent, and adherence to those components to be public. This enables entities to rely on the business or technical process carried out by others with trust and confidence. If done correctly, a trust framework is invisible to those who rely on it every day. It allows individuals and entities to conduct digital transactions knowing that the trust frameworks underpin, create accountability, and support the decisions they’re making.

Use Cases for Trust Frameworks

Historically speaking, trust frameworks have been extraordinarily complex and only worth the investment for high-value, high-volume transactions, such as the ones established by credit card companies. Now, with the introduction of decentralized technologies, there is a need to create digital trust frameworks that work for a much broader variety of transactions. Realizing the scope of this work comes with the recognition that there will be many different trust frameworks, both small and large in scope, for different federations across the web. Given that context, it is important to preserve end-user agency as much as possible as trust frameworks are developed and adoption and mutual recognition increases.

Looking at the ecosystem today, we can broadly group trust frameworks into three categories:

Domain-specific Trust Frameworks These are typically developed to serve a specific use-case, for example within a particular industry Often driven by industry and/or NGOs These have been able to develop faster than national trust frameworks (which are based in legislation), and as such may inform the development of national trust frameworks National Trust Frameworks Typically broad in application and to facilitate a policy objective (for example, increased trust in data sharing) Driven by individual governments to address the needs of their citizens and residents Based in legislation, with more enforcement powers than either Domain-specific Trust Frameworks or International Trust Frameworks Likely to be informed by both Domain-specific Trust Frameworks and International Trust Frameworks International Trust Frameworks These are typically broad in nature and developed to serve many countries, much like a model law Typically driven by governments, industry, or NGOs but geographically agnostic Likely to inform National Trust Frameworks Accreditation and Assurance

An important part of satisfying the operational components of a trust framework is the ability to accredit ecosystem participants against the trust framework. This is a logical extension of the rules, requirements, and regulations trust frameworks set out. Trust frameworks typically include an accreditation scheme and associated ongoing compliance testing.

One aspect of accreditation in the identity context is compliance with standards. In the context of identity related trust frameworks, there are several kinds of assurance that relying parties will typically seek. These can include binding, information, authentication, and federation and identity assurance. Each standard may define their own distinct levels of assurance. The NIST Digital Identity Requirements and New Zealand Identification Management Standards are a good example of how this works in practice.

The process of accreditation and a successful certification is a core part of trust frameworks as it proves to the wider ecosystem (including auditors) that the entity, solution, or piece of software meets the business and technical requirements defined. Digital identity systems are increasingly modular, and one solution might involve a variety of different components, roles and providers. These should be developed and defined as part of the process of standing up a trust framework, testing its capabilities and defining processes around accreditation.

Technical Interoperability

Trust frameworks help to improve interoperability between entities by defining a common set of operating rules. In addition to setting out business and legal rules, it is important that high level technical rules are specified as well. Trust frameworks must clearly define expectations around the technical standards to be used, as well as what aspects of these standards are normatively required, optional, or somewhere in between. When it comes to digital identity trust frameworks, this may mean building on open-source code or evaluating against open test suites.

Test suites allow for normative testing around standards requirements and offer a way for parties to audit and ensure the processes being used throughout the identity lifecycle. They can be incredibly useful not only for entities using the trust framework, but for mutually recognized trust frameworks to understand and interpret the requirements coming from a particular set of rules.

Ongoing development of several digital identity trust frameworks based on the emerging decentralized web of trust can be found at industry organizations such as the Kantara Initiative and Trust Over IP Foundation as well as government-driven initiatives such as the Pan-Canadian Trust Framework.

Learn Concepts: Trust Frameworks was originally published in MATTR on Medium, where people are continuing the conversation by highlighting and responding to this story.


Learn Concepts: Selective Disclosure

An important principle that we want to achieve when designing any system that involves handling Personally Identifiable Information (PII) is to minimize the data disclosed in a given interaction. When users share information, they should be able to choose what and how much they share on a case-by-case basis, while the relying parties receiving the information must be able to maintain assurances ab

An important principle that we want to achieve when designing any system that involves handling Personally Identifiable Information (PII) is to minimize the data disclosed in a given interaction. When users share information, they should be able to choose what and how much they share on a case-by-case basis, while the relying parties receiving the information must be able to maintain assurances about the presented information’s origin and integrity. This process is often referred to as selective disclosure of data. As technologists, by having solutions that easily achieve selective disclosure, we can drive a culture based on the minimum information exchange required to enhance user privacy.

Privacy and Correlation

Selective disclosure of information is particularly relevant when evaluating approaches to using verifiable credentials (VCs). Because authorities are able to issue credentials to a subject’s digital wallet, the subject is able to manage which data they disclose to relying parties as well as how that disclosure is performed. This presents an opportunity for those designing digital wallets to consider the user experience of data disclosure, particularly as it relates to the underlying technology and cryptography being used for data sharing.

The problem of user privacy as it relates to digital identity is a deep and complicated one, however the basic approach has been to allow users to share only the information which is strictly necessary in a particular context. The VC Data Model spec provides some guidance on how to do so, but stops short of offering a solution to the issue of managing user privacy and preventing correlation of their activities across different interactions:

Organizations providing software to holders should strive to identify fields in verifiable credentials containing information that could be used to correlate individuals and warn holders when this information is shared.

A number of different solutions have been deployed to address the underlying concerns around selective disclosure. Each solution makes a different set of assumptions and offers different tradeoffs when it comes to usability and convenience.

Approaches to Selective Disclosure

When it comes to solutions for selective disclosure of verifiable credentials, there are many different ways to tackle this problem, but three of the most common are:

Just in time issuance — contact the issuer at request time either directly or indirectly for a tailored assertion Trusted witness — use a trusted witness between the provider and the relying party to mediate the information disclosure Cryptographic solutions — use a cryptographic technique to disclose a subset of information from a larger assertion Just in time issuance

Just in time issuance, a model made popular by OpenID Connect, assumes the issuer is highly available, which imposes an infrastructure burden on the issuer that is proportional to the number of subjects they have information for and where those subjects use their information. Furthermore, in most instances of this model, the issuer learns where a subject is using their identity information, which can be a serious privacy problem.

Trusted witness

Trusted witness shifts this problem to be more of a presentation concern, where a witness de-anonymizes the subject presenting the information and presents an assertion with only the information required by the relying party. Again, this model requires a highly available party other than the holder and relying party present when a subject wants to present information, one that must be highly trusted and one that bears witness to a lot of PII on the subject, leading to privacy concerns.

Cryptographic solutions

Cryptographic solutions offer an alternative to these approaches by solving the selective disclosure problem directly at the core data model layer of the VC, providing a simpler and more flexible method of preserving user privacy.

There are a variety of ways that cryptography can be used to achieve selective disclosure or data minimization, but perhaps the most popular approach is using a branch of cryptography often known as Zero-Knowledge Proofs, or ZKPs. The emergent feature of this technology is that a prover can prove knowledge of some data without exposing any additional data. Zero-knowledge proofs can be achieved in a flexible manner with verifiable credentials using multi-message digital signatures such as BBS+.

Traditional Digital Signatures

Traditional digital signatures look a bit like this. You have a message (virtually any kind of data for which you want to establish integrity) and a keypair (private and public key) which you use to produce a digital signature on the data. By having the message, public key, and the signature; verifiers are able to evaluate whether the signature is valid or not, thereby establishing the integrity of the message and the authenticity of the entity that signed the message. In the context of verifiable credentials, the entity doing the signing is the issuer of the credential, while the entity doing the verification is the verifier. The keypair in question belongs to the issuer of the credential, which allows verifiers to establish the authority on that credential in a verifiable manner.

Sign Verify Multi-message Digital Signatures

Multi-message digital signature schemes (like BBS+), on the other hand, are able to sign an array of messages, rather than a single message over which the entire digital signature is applied. The same mechanism is used wherein a private key produces a digital signature over the messages you wish to sign, but now you have the flexibility of being able to break a message up into its fundamental attributes. In the context of verifiable credentials, each message corresponds to a claim in the credential. This presents an opportunity for selective disclosure due to the ability to derive and verify a proof of the digital signature over a subset of messages or credential attributes.

Sign Verify

In addition to the simple ability to sign and verify a set of messages, multi-message digital signatures have the added capability of being able to derive a proof of the digital signature. In the context of verifiable credentials, the entity deriving the proof is the credential subject or holder. This process allows you to select which messages you wish to disclose in the proof and which messages you want to keep hidden. The derived proof indicates to the verifier that you know all of the messages that have been signed, but that you are only electing to disclose a subset of these messages.

Derive Proof Verify Proof

The verifier, or the entity with which you’re sharing the data, is only able to see the messages or credential claims which you have selectively disclosed to them. They are still able to verify the integrity of the messages being signed, as well as establish the authenticity of the issuer that originally signed the messages. This provides a number of privacy guarantees to the data subject because relying parties are only evaluating the proof of the signature rather than the signature itself.

Learn Concepts: Selective Disclosure was originally published in MATTR on Medium, where people are continuing the conversation by highlighting and responding to this story.


Learn Concepts: Semantic Web

With so much data being created and shared on the internet, one of the oldest challenges in building digital infrastructure has been how to consistently establish meaning and context to this data. The semantic web is a set of technologies whose goal is to make all data on the web machine-readable. Its usage allows for a shared understanding around data that enables a variety of real-world applicat

With so much data being created and shared on the internet, one of the oldest challenges in building digital infrastructure has been how to consistently establish meaning and context to this data. The semantic web is a set of technologies whose goal is to make all data on the web machine-readable. Its usage allows for a shared understanding around data that enables a variety of real-world applications and use cases.

The challenges to address with the semantic web include:

vastness — the internet contains billions of pages, and existing technology has not yet been able to eliminate all semantically duplicated terms vagueness — imprecise concepts like ‘young’ or ‘tall’ make it challenging to combine different knowledge bases with overlapping but subtly different concepts uncertainty — precise concepts with uncertain values can be hard to reason about, this mirrors the ambiguity and probabilistic nature of everyday life inconsistency — logical contradictions create situations where reasoning breaks down deceit — intentionally misleading information spread by bad actors, can be mitigated with cryptography to establish information integrity Linked Data

Linked data is the theory behind much of the semantic web effort. It describes a general mechanism for publishing structured data on the internet using vocabularies like schema.org that can be connected together and interpreted by machines. Using linked data, statements encoded in triples (subject → predicate → object) can be spread across different websites in a standard way. These statements form the substrate of knowledge that spans across the entire internet. The reality is that the bulk of useful information on the internet today is unstructured data, or data that is not organized in a way which makes it useful to anyone beyond the creators of that data. This is fine for the cases where data remains in a single context throughout its lifecycle, but it becomes problematic when trying to share data across contexts while retaining its semantic meaning. The vision for linked data is for the internet to become a kind of global database where all data can be represented and understood in a similar way.

One of the biggest challenges to realizing the vision of the internet as a global database is enabling a common set of underlying semantics that can be consumed by all this data. A proliferation of data becomes much less useful if the data is redundant, unorganized, or otherwise messy and complicated. Ultimately, we need to double down on the usage of common data vocabularies and common data schemas. Common data schemas combined with the security features of verifiable data will make fraud more difficult, making it easier to transmit and consume data so that trust-based decisions can be made. Moreover, the proliferation of common data vocabularies will help make data portability a reality, allowing data to be moved across contexts while retaining the semantics of its original context.

Semantic Web Technologies

The work around developing semantic web technology has been happening for a very long time. The vision for the semantic web has been remarkably consistent throughout its evolution, although the specifics around how to accomplish this and at what layer has developed over the years. W3C’s semantic web stack offers an overview of these foundational technologies and the function of each component in the stack.

The ultimate goal of the semantic web of data is to enable computers to do more useful work and to develop systems that can support trusted interactions over the network. The shared architecture as defined by the W3C supports the ability for the internet to become a global database based on linked data. Semantic Web technologies enable people to create data stores on the web, build vocabularies, and write rules for handling data. Linked data are empowered by technologies such as RDF, SPARQL, OWL, and SKOS.

RDF provides the foundation for publishing and linking your data. It’s a standard data model for representing information resources on the internet and describing the relationships between data and other pieces of information in a graph format. OWL is a language which is used to build data vocabularies, or “ontologies”, that represent rich knowledge or logic. SKOS is a standard way to represent knowledge organization systems such as classification systems in RDF. SPARQL is the query language for the Semantic Web; it is able to retrieve and manipulate data stored in an RDF graph. Query languages go hand-in-hand with databases. If the Semantic Web is viewed as a global database, then it is easy to understand why one would need a query language for that data.

By enriching data with additional context and meaning, more people (and machines) can understand and use that data to greater effect.

JSON-LD

JSON-LD is a serialization format that extends JSON to support linked data, enabling the sharing and discovery of data in web-based environments. Its purpose is to be isomorphic to RDF, which has broad usability across the web and supports additional technologies for querying and language classification. RDF has been used to manage industry ontologies for the last couple decades, so creating a representation in JSON is incredibly useful in certain applications such as those found in the context of Verifiable Credentials (VCs).

The Linked Data Proofs representation of Verifiable Credentials makes use of a simple security protocol which is native to JSON-LD. The primary benefit of the JSON-LD format used by LD-Proofs is that it builds on a common set of semantics that allow for broader ecosystem interoperability of issued credentials. It provides a standard vocabulary that makes data in a credential more portable as well as easy to consume and understand across different contexts. In order to create a crawl-able web of verifiable data, it’s important that we prioritize strong reuse of data schemas as a key driver of interoperability efforts. Without it, we risk building a system where many different data schemas are used to represent the same exact information, creating the kinds of data silos that we see on the majority of the internet today. JSON-LD makes semantics a first-class principle and is therefore a solid basis for constructing VC implementations.

JSON-LD is also widely adopted on the web today, with W3C reporting it is used by 30% of the web and Google making it the de facto technology for search engine optimization. When it comes to Verifiable Credentials, it’s advantageous to extend and integrate the work around VCs with the existing burgeoning ecosystem of linked data.

Learn Concepts: Semantic Web was originally published in MATTR on Medium, where people are continuing the conversation by highlighting and responding to this story.


Learn Concepts: Verifiable Data

The ability to prove the integrity and authenticity of shared data is a key component to establishing trust online. Given that we produce so much data and are constantly sharing and moving that data around, it is a complex task to identify a solution that will work for the vast majority of internet users across a variety of different contexts. The fundamental problem to address is how to establis

The ability to prove the integrity and authenticity of shared data is a key component to establishing trust online. Given that we produce so much data and are constantly sharing and moving that data around, it is a complex task to identify a solution that will work for the vast majority of internet users across a variety of different contexts.

The fundamental problem to address is how to establish authority on a piece of data, and how to enable mechanisms to trust those authorities in a broad set of contexts. Solving this problem on a basic level allows entities to have greater trust in the data they’re sharing, and for relying parties to understand the integrity and authenticity of the data being shared.

We use the overarching term verifiable data to refer to this problem domain. Verifiable data can be further expanded into three key pillars:

Verifiable data Verifiable relationships Verifiable processes Verifiable data

This refers to the authenticity and integrity of the actual data elements being shared.

Verifiable relationships

This refers to the ability to audit and understand the connections between various entities as well as how each of these entities are represented in data.

Verifiable processes

This describe the ability to verify any digital process such as onboarding a user or managing a bank account (particularly with respect to how data enables the process to be managed and maintained).

These closely-related, interdependent concepts rely on verifiable data technology becoming a reality.

Verifiable Credentials

The basic data model of W3C Verifiable Credentials may be familiar to developers and architects that are used to working with attribute-based credentials and data technologies. The issuer, or the authority on some information about a subject (e.g. a person), issues a credential containing this information in the form of claims to a holder. The holder is responsible for storing and managing that credential, and in most instances uses a piece of software that acts on their behalf, such as a digital wallet. When a verifier (sometimes referred to as a relying party) needs to validate some information, they can request from the holder some data to meet their verification requirements. The holder unilaterally determines if they wish to act upon the request and is free to present the claims contained in their verifiable credentials using any number of techniques to preserve their privacy.

Verifiable Credentials form the foundation for verifiable data in the emerging web of trust. They can be thought of as a container for many different types of information as well as different types of credentials. Because it is an open standard at the W3C, verifiable credentials are able to widely implemented by many different software providers, institutions, governments, and businesses. Due to the wide applicability of these standards, similar content integrity protections and guarantees are provided regardless of the implementation.

Semantics and Schemas

The authenticity and integrity-providing mechanisms presented by Verifiable Credentials provide additional benefits beyond the evaluation of verifiable data. They also provide a number of extensibility mechanisms that allow data to be linked to other kinds of data in order to be more easily understood in the context of relationships and processes.

One concrete example of this is the application of data schemas or data vocabularies. Schemas are a set of types and properties that are used to describe data. In the context of data sharing, schemas are an incredibly useful and necessary tool in order to represent data accurately from the point of creation to sharing and verification. In essence, data schemas in the Verifiable Credential ecosystem are only useful if they are strongly reused by many different parties. If each implementer of Verifiable Credentials chooses to describe and represent data in a slightly different way, it creates incoherence and inconsistency in data and threatens to diminish the potential of ubiquitous adoption of open standards and schemas.

Verifiable Credentials make use of JSON-LD to extend the data model to support dynamic data vocabularies and schemas. This allows us to not only use existing JSON-LD schemas, but to utilize the mechanism defined by JSON-LD to create and share new schemas as well. To a large extent this is what JSON-LD was designed for; the adoption and reuse of common data vocabularies.

This type of Verifiable Credential is best characterized as a kind of Linked Data Proof. It allows issuers to make statements that can be shared without loss of trust because their authorship can be verified by a third party. Linked Data Proofs define the capability for verifying the authenticity and integrity of Linked Data documents with mathematical proofs and asymmetric cryptography. It provides a simple security protocol which is native to JSON-LD. Due to the nature of linked data, they are built to compactly represent proof chains and allow a Verifiable Credential to be easily protected on a more granular basis; on a per-attribute basis rather than a per-credential basis.

This mechanism becomes particularly useful when evaluating a chain of trusted credentials belonging to organizations and individuals. A proof chain is used when the same data needs to be signed by multiple entities and the order in which the proofs were generated matters. For example, such as in the case of a notary counter-signing a proof that had been created on a document. Where order needs to be preserved, a proof chain is represented by including an ordered list of proofs with a “proof chain” key in a Verifiable Credential. This kind of embedded proof can be used to establish the integrity of verifiable data chains.

Overall, the ability for data to be shared across contexts whilst retaining its integrity and semantics is a critical building block of the emerging web of trust.

Learn Concepts: Verifiable Data was originally published in MATTR on Medium, where people are continuing the conversation by highlighting and responding to this story.


Learn Concepts: Digital Wallets

In order to coordinate the authentication needs of apps and services on the web, many of today’s users will leverage services such as password managers. These tools help users keep track of how they’ve identified themselves in different contexts and simplify the login process for different services. In many ways, the need to overlay such services in order to preserve non-negotiable security proper

In order to coordinate the authentication needs of apps and services on the web, many of today’s users will leverage services such as password managers. These tools help users keep track of how they’ve identified themselves in different contexts and simplify the login process for different services. In many ways, the need to overlay such services in order to preserve non-negotiable security properties reflects the broken state of identity on the internet today. Users of these apps (i.e. the data subjects) are often an afterthought when a trust relationship is established between data authorities and apps or services consuming and relying on user data.

Asymmetry in the nature of the relationships between participants largely prevents users from asserting their data rights as subjects of the data. Users are left to deal with the problems inherent in such a model, foisting upon them the responsibility of implementing appropriate solutions to patch over the shortcomings of identity management under this legacy model.

The emerging web of trust based upon self-certifying identifiers and user-centric cryptography is shifting this fundamental relationship by refashioning the role of the user. This role (known in the VC data model as a “holder”) is made central to the ecosystem and, importantly, on equal footing with the issuers of identity-related information and the relying parties who require that data to support their applications and services.

The reframing of the user as a first-class citizen and their empowerment as ‘holder’ represents a shift towards a new paradigm. Such a paradigm offers users greater sovereignty of their own information and empowerment to manage their digital identity. Users are able to exercise their new role in this ecosystem by utilizing a new class of software known as digital wallets.

Digital wallets are applications that allow an end user to manage their digital credentials and associated cryptographic keys. They allow users to prove identity-related information about themselves and, where it’s supported, choose to selectively disclose particular attributes of their credentials in a privacy-preserving manner.

Wallets and Agents

When working with technology standards that are inherently decentralized, it’s important to establish a common context and consensus in our choice of terminology and language. Convergence on key terms that are being used to describe concepts within the emerging decentralized identity and self-sovereign identity technologies allows participants to reach a shared understanding. Consequently, participating vendors are able to understand how they fit into the puzzle and interoperability between vendor implementations is made possible.

Through dedicated research and careful coordination with the broader technical community, the Glossary Project at DIF offers a useful definition for both wallets and agents.

Wallets
Provide storage of keys, credentials, and secrets, often facilitated or controlled by an agent.
Agents
An agent is a software representative of a subject (most often a person) that controls access to a wallet and other storage, can live in different locations on a network (cloud vs. local), and can facilitate or perform messaging or interactions with other subjects.

The two concepts are closely related, and are often used interchangeably. In short, the Glossary Project found that an agent is most commonly a piece of software that lets you work with and connect to wallets. Wallets can be simple, while agents tend to be more complex. Agents often need access to a wallet in order to retrieve credentials, keys, and/or messages that are stored there.

At MATTR, we tend to use the terms ‘digital wallet’ or simply ‘wallet’ to holistically describe the software that is utilized by end-users from within their mobile devices, web browsers, or other such user-controlled devices or environments. A digital wallet can be thought of as a kind of agent, though we try to make the distinction between the software that sits on a user’s device and the data managed and logic facilitated by a cloud-based platform in support of the wallet’s capabilities. We like the term ‘wallet’ because it is analogous to real-world concepts that by and large parallel the primary function of a wallet; to store and retrieve identity-related information.

User-centric Design

As end users have often found themselves the casualty of the information systems used by the modern web, there has been little opportunity to allow users to directly manage their data and negotiate what data they wish to withhold or disclose to certain parties. Under the new web of trust paradigm, the rights of the data subject are codified in standards, processes, and protocols guaranteeing the user the power to exercise agency. The interjection of the wallet to support end-users as data subjects on equal footing with issuers of identity information and relying parties provides an indispensable conduit and control point for this information that enables new opportunities for user-centric design.

The innovation in this area is only just beginning and there is no limit to the kinds of new experiences application developers can design and deliver to users. Some examples include:

Allowing users to synchronize their data across multiple applications Allowing users to self-attest to a piece of data or attest to data self-asserted by peers Allowing a user to explicitly give consent around how their data may be used Allowing users to revoke their consent for access to the continued use of and/or persistence of a particular piece of data Allowing users to opt-in to be discoverable to other verified users, provided they can mutually verify particular claims and attributes about themselves Allowing users to opt-in to be discoverable to certain service providers and relying parties, provided they can mutually verify particular claims and attributes about themselves

These are just a handful of the potential ways that developers can innovate to implement user-centric experiences. MATTR offers the tools necessary to create new kinds of wallet and authentication experiences for users and we’re excited to see what developers come up with when given the opportunity to create applications and services inspired by these new standards and technologies.

Learn Concepts: Digital Wallets was originally published in MATTR on Medium, where people are continuing the conversation by highlighting and responding to this story.


Learn Concepts: Web of Trust 101

The original vision for the World Wide Web was an open platform on which everyone could freely communicate and access information. It was built on the decentralized architecture of the internet, used open standards, and functioned as an accessible platform that would inherit and amplify the fundamentally decentralized nature of the network that underpinned it. However, the reality today has falle

The original vision for the World Wide Web was an open platform on which everyone could freely communicate and access information. It was built on the decentralized architecture of the internet, used open standards, and functioned as an accessible platform that would inherit and amplify the fundamentally decentralized nature of the network that underpinned it.

However, the reality today has fallen far short of its founding vision. The modern internet is largely centralized and siloed. The vast majority of web traffic belongs to a few powerful corporations that control the distribution of data through platforms designed to selectively serve up information based on in-depth analysis of their users’ data. The lack of an identity system native to the internet over time has created an imbalance of power that erodes users’ digital rights.

Several decades after the web was introduced, most of us are now accustomed to widespread spam, fraud, abuse, and misinformation. We don’t have any real agency over how our data is used, and the corporations controlling our data have shown their inability to properly shoulder the responsibility that comes with it. We’re locked into this system, with no reasonable ability to opt out.

As a result, the modern internet has made it incredibly difficult to establish trust with others online, creating many barriers to participation that often leave everyday users out of the value chain. Information and data, and the value they create, are no longer freely accessible by the users creating it — most of whom are utterly unaware of the limited agency they have in accessing it. To fix this fundamental problem of digital trust, we need to begin by building a system that allows users to control their identities and to move their personal data freely from one online platform to another without fear of vendor lock-in.

Evolution of Digital Trust

The emerging “Web of Trust” is an idea that has been around since the dawn of the internet. To explain what motivated its creation, let’s take a look at how trust on the internet functions today.

Though we may not always be aware, we rely on a basic form of security practically every day we use the internet. HTTPS, the secure browsing protocol for the World Wide Web, uses a common infrastructure based on digital signatures to allow users to authenticate and access websites, and protect the privacy and integrity of the data exchanged while in transit. It is used to establish trust on all types of websites, to secure accounts, and to keep user communications, identity, and web browsing private.

Centralized PKI System

This is all based on the usage of cryptographic keys, instead of passwords, to perform security and encryption. Public key cryptography is a cryptographic technique that enables entities to securely communicate on an insecure public network (the internet), and reliably verify the identity of users via digital signatures. It is required for activities where simple passwords are an inadequate authentication method and more rigorous proof is required to confirm the identity of the parties involved in the communication and to validate the information being transferred.

The type of Public Key Infrastructure (PKI) currently used by the internet primarily relies on a hierarchical system of certificate authorities (CAs), which are effectively third-parties that have been designated to manage identifiers and public keys. Virtually all internet software now relies on these authorities. Certificate authorities are responsible for verifying the authenticity and integrity of public keys that belong to a given user, all the way up to a ‘self-signed’ root certificate. Root certifications are typically distributed with applications such as browsers and email clients. Applications commonly include over one hundred root certificates from dozens of PKIs, thereby bestowing trust throughout the hierarchy of certificates which lead back to them. The concept is that if you can trust the chain of keys, you can effectively establish secure communication with another entity with a reasonable level of assurance that you’re talking to the right person.

However, the reliance on certificate authorities creates a centralized dependency for practically all transactions on the internet that require trust. This primarily has to do with the fact that current PKI systems tightly control who gets to manage and control the cryptographic keys associated with certificates. This constraint means that modern cryptography is largely unusable for the average user, forcing us to borrow or ‘rent’ identifiers such as our email addresses, usernames, and website domains through systems like DNS, X.509, and social networks. And because we need these identities to communicate and transact online, we’re effectively beholden to these systems which are outside of our control. In addition, the usability challenges associated with current PKI systems mean that much of Web traffic today is unsigned and unencrypted, such as on major social networks. In other words, cryptographic trust is the backbone of all internet communications, but that trust rarely trickles down to the user level.

A fully realized web of trust instead relies on self-signed certificates and third party attestations, forming the basis for what’s known as a Decentralized Public Key Infrastructure (DPKI). DPKI returns control of online identities to the entities they belong to, bringing the power of cryptography to everyday users (we call this user-centric cryptography) by delegating the responsibility of public key management to secure decentralized datastores, so anyone and anything can start building trust on the web.

A Trust Layer for the Internet

The foundational technology for a new DPKI is a system of distributed identifiers for people, organizations, and things. Decentralized identifiers are self-certifying identifiers that allow for distributed discovery of public keys. DIDs can be stored on a variety of different data registries, such as blockchains and public databases, and users can always be sure that they’re talking to the right person or entity because an identifier’s lookup value is linked to the most current public keys for that identifier. This creates a kind of even playing field where the standards and requirements for key management are uniform across different users in an ecosystem, from everyday users to large corporations and everything in between.

Decentralized PKI System

This will, in the first place, give users far greater control over the manner in which their personal data is being used by businesses, allowing them to tweak their own experience with services to arrive at that specific trade-off between convenience and data protection that best suits their individual requirements. But more importantly, it will allow users to continue to federate data storage across multiple services while still delivering the benefits that come from cross-platform data exchange. In other words, it gives them the ability to manage all their data in the same way while being able to deal with data differently depending on the context they are in. This also allows them to move their personal data freely from one online platform to another without losing access to the services they need, and without fear of vendor lock-in.

Eventually, this will allow for portability not only of data but of the trust and reputation associated with the subjects of that data. For instance, a user might be able to transfer their reputation score from one ride-sharing service to another, or perhaps use the trust they’ve established in one context in another context entirely.

This emerging decentralized web of trust is being forged by a global community of developers, architects, engineers, organizations, hackers, lawyers, activists, and more working to push forward and develop web standards for things like credential exchange, secure messaging, secure storage, and trust frameworks to support this new paradigm. The work is happening in places like the World Wide Web Foundation, W3C Credentials Community Group, Decentralized Identity Foundation, Trust Over IP Foundation, Linux Foundation’s Hyperledger project, and Internet Engineering Task Force, to name a few.

Learn Concepts: Web of Trust 101 was originally published in MATTR on Medium, where people are continuing the conversation by highlighting and responding to this story.

Tuesday, 22. December 2020

Aaron Parecki

Learn OAuth over the winter break!

Over the last year, I've helped thousands of software developers learn about OAuth by hosting live and virtual workshops, and all this knowledge is now available as an on-demand video course!

If you've been putting off setting aside some time to learn about OAuth, now is your chance!

Over the last year, I've helped thousands of developers learn about OAuth by hosting live workshops in person, online events through O'Reilly and Okta, as well as by publishing videos on YouTube! I'm super thrilled to announce that I just finished packaging up the workshop and have launched it as a new course, "The Nuts and Bolts of OAuth 2.0"!

The course is 3.5 hours of video content, quizzes, as well as interactive exercises with a guided learning tool to get you quickly up to speed on OAuth, OpenID Connect, PKCE, best practices, and tips for protecting APIs with OAuth.

The course is available now on Udemy, and if your company has a Udemy for Business subscription you can find it there as well! If you download the app, you can even sync the video course to a mobile device to watch everything offline!

The exercises in the course will walk you through the various OAuth flows to set up an OAuth server, get an access token, use a refresh token, and learn the user's name and email with OpenID Connect. You can see a sneak peek of the tool that interactively helps you debug your apps at oauth.school.

Free OAuth Videos

I've also got a bunch of videos about OAuth available on YouTube for you to watch at any time! Take a look at my curated playlist of videos where you'll find everything from live sketch notes of a conversation about PKCE and OAuth security, to a description of OAuth phishing, to details on why you shouldn't use the implicit flow.

Tuesday, 22. December 2020

Matt Flynn: InfoSec | IAM

Oracle Strengthens Interoperability and User Experience with General Availability of FIDO2 WebAuthn Support for Cloud Identity

"Given the distributed nature of today’s technology environment, zero trust has become the standard for security. Every interaction must be authenticated and validated for every user accessing every system or application every time. To that end, interoperability is more important than ever.To that end, interoperability is more important than ever. FIDO2 Web Authentication (WebAuthn) is quickly eme

"Given the distributed nature of today’s technology environment, zero trust has become the standard for security. Every interaction must be authenticated and validated for every user accessing every system or application every time. To that end, interoperability is more important than ever.To that end, interoperability is more important than ever. FIDO2 Web Authentication (WebAuthn) is quickly emerging as an important interoperability standard that enables users to select and manage an authenticator of their own (security keys, or built-in platform authenticators, such as a mobile device) that works with their web browser of choice (Google Chrome, Mozilla Firefox, Microsoft Edge, Apple Safari, etc.) for secure access to any websites or applications that support the WebAuthn standard."

"Oracle is happy to announce the general availability of FIDO2 WebAuthn for our cloud identity service. This means that websites and applications that are protected by Oracle can enable their audience of users to authenticate with FIDO2 authenticators for multi-factor authentication (MFA) as well as passwordless authentication. This simplifies the user experience and may reduce the number of authenticators that users need to access the variety of web applications they interact with on a regular basis. Ultimately, this gives users more choice, more control, and a frictionless user experience.

Read more on the Oracle Cloud Security Blog > Oracle Strengthens Interoperability and User Experience with General Availability of FIDO2 WebAuthn Support for Cloud Identity.

Tuesday, 22. December 2020

Hans Zandbelt

MVP OAuth 2.x / OpenID Connect modules for NGINX and Apache

Last year I wrote about a new development of a generic C-library for OAuth 2.x / OpenID Connect https://hanszandbelt.wordpress.com/2019/03/22/oauth-2-0-and-openid-connect-libraries-for-c/. It’s taken a bit longer than anticipated – due to various circumstances – but there’s now a collection of modules that … Continue reading →

Last year I wrote about a new development of a generic C-library for OAuth 2.x / OpenID Connect https://hanszandbelt.wordpress.com/2019/03/22/oauth-2-0-and-openid-connect-libraries-for-c/.

It’s taken a bit longer than anticipated – due to various circumstances – but there’s now a collection of modules that is at the Minimum Viable Product stage i.e. the code is stable and production quality, but it is limited in its feature set.

mod_oauth2 – OAuth 2.0 Resource Server module for Apache ngx_oauth2_module – OAuth 2.0 Resource Server module for NGINX mod_sts – Security Token Exchange module for Apache ngx_sts_module – Security Token Exchange module for NGINX ngx_openidc_module – OpenID Connect Relying Party module for NGINX

Enjoy!


SSI Ambassador

The trust infrastructure of self-sovereign identity ecosystems.

The trust infrastructure is concerned with the question of how and why the presented information can be trusted. It defines the rules for all stakeholders and enables legally binding relationships with the combination of governance frameworks, which are built on top of trust frameworks. But before we dive deeper into the part of the trust infrastructure, we need to understand the core components

The trust infrastructure is concerned with the question of how and why the presented information can be trusted. It defines the rules for all stakeholders and enables legally binding relationships with the combination of governance frameworks, which are built on top of trust frameworks.

But before we dive deeper into the part of the trust infrastructure, we need to understand the core components and the different types of identity architecture first.

Core components of identity architecture

There are three core components within an identity system, which in general mainly manage relationships. These are identifiers enabling the means for “remembering, recognizing, and relying on the other parties to the relationship” as explained by Phil Windley. In the case of SSI, these are decentralized Identifiers (DIDs), which are created by a controller, which might be a person, organization or software system. Controllers can use different authentication factors, which according to the Guidance for the application of the level of assurance for eIDAS (CEF DIGITAL), can be possession-based factors (e.g. hardware), knowledge-based factors (e.g. keys or passwords) or inherent factors (e.g. biometrics like a fingerprint). Oftentimes a combination of different authentication factors is used to demonstrate the authority of an identifier.

Architectural types of identity systems; Image adapted, original by Phil Windley

Phil Windley distinguishes between administrative, algorithmic and autonomic systems, which are illustrated in the image above. We are of course quite familiar with administrative systems such as e-mail, mobile or social network services. Algorithmic systems in contrast leverage some sort of distributed ledger as verified data registry to register a public key / decentralized identifier on a ledger. This gives the controller more sovereignty of the identifier among other perks. However, due to privacy concerns, keys or identifiers of individuals should not be stored on a publicly accessible database. Hence, autonomic identity architecture was developed to enable the “controller to use her private key to authoritatively and non-repudiably [sic] sign statements about the operations on the keys and their binding to the identifier, storing those in an ordered key event log” as Phil Windley states. Current SSI implementations tend to use a combination of algorithmic and autonomic architecture.

Governance frameworks

The Business Dictionary defines governance as the “establishment of policies, and continuous monitoring of their proper implementation, by the members of the governing body.” It includes the mechanisms required to balance powers and defines their primary duties to enhance the prosperity and viability of the organization. The objective for governance entities is to ensure the alignment of involved stakeholders, the definition of the implementation and the processes and use-cases executed on top of it. The purpose of a governance framework is to define the different stakeholders, determine their rights and duties as well as defining the policies under which the network is operated. Therefore, it serves as a legal foundation for the operation of the particular network. It consists of several legal documents, which are published by the governing authority.

The governance of the network (the verifiable data registry also referred to as ledger or trust anchor) itself is only a small part of the total governance required. According to the Trust over IP (ToIP) foundation there are four layers, which require an adapted governance framework matching the needs of the particular layer.

The Trust over IP (ToIP) Stack; Image source: Trust over IP Foundation

As illustrated in the figure above, the Trust over IP stack is not only separated in layers, but also in technical and governance stacks within the layers. The stack is indented to provide the certainty for higher levels that the underlying ones can be trusted.

Layer one includes the verifiable data registries, which can be implemented on different technology frameworks. The Sovrin Foundation is an example of a governance authority, which published a governance framework for layer one. Other public utilities include IDunion, Indicio and Bedrock among others. While the mentioned networks align their efforts particularly to the ToIP stack there are countless others, which can be used as a public utility as the W3C DID specification registry discloses. These range from generic public permissionless networks such as Ethereum or Bitcoin to networks with permissioned write access, which serve a particular use-case.

Tim Bouma (Senior policy analyst for identity management at the treasury board secretariat of Canada) does not see the need of the government to build and operate a verifiable data registry and highlights the importance of a plurality of operators. However, he points out that the involvement and participation of governments is crucial in defining how the infrastructure is used and relied on as stated in a personal interview.

DID methods as specified in the W3C DID specification registry.

The second layer describes the communication between agents. Within the ToIP stack, this communication is indented to be executed via a hybrid of an algorithmic and atomic architecture such as peer DIDs or KERI implementations of self-certifying identifiers as described by Ph.D. Samuel M. Smith. This means that publicly resolvable DIDs are used for public entities and private peer DIDs for individuals. The following illustration provided by Lissi provides an overview of these interactions.

SSI interactions and the usage of public and peer DIDs; Image source: Lissi

However, not all SSI implementations use peer DIDs. For instance, the ESSIF-MVP1 does not currently use peer DIDs but might add them later as deemed appropriate according to the technical specification of DID modelling. Hence, the same type of DID is used for both issuer and holder. The governing authority of layer two is highly dependent on the communication protocols used by the implementation in question. For implementations, which use peer DIDs according to the DIDcomm protocol the governing entity is the DIDcomm working group at the Decentralized Identity Foundation (DIF). Both layer one and layer two define technical or rather cryptographic trust, in contrast to layer three and four, which define human trust.

Layer three protocols support the exchange of data such as verified credentials with different types of signatures, which enable a holder to create verifiable presentations as explained in the ToIP Aries-RFC. It states that one of the goals of the ToIP Foundation is “to standardize all supported credential exchange protocols so that any ToIP-compatible agent, wallet, and secure data store can work with any other agent, wallet, and secure data store.” An issuer can issue any set of claims to any holder, which can then prove them to any verifier. The verifier can decide, which issuers and which claims it trusts.

Layer four of the stack defines the rules for a particular digital trust ecosystem such as healthcare, finance, food products, education etc. These are led by a governance authority, which already exists or is established for this particular purpose. It consists of a variety of different stakeholders such as business entities, government agencies, individuals or other interested parties. These ecosystem frameworks also define the semantics of verified credentials. The semantic of a verified credential defines, which attributes are part of it and their meaning in the particular context. If you want to join an existing ecosystem or want to know more about their work you can find the public ToIP confluence documentary hub here.

Trust frameworks

A trust framework sets the overall legal framework for digital interactions. These trust frameworks are technology agnostic and are uniquely adapted to the jurisdiction they serve. They set the rules for the recognition of electronic identification and authentication and specify the requirements to achieve a certain level of assurance (LoA).

The combination of the different governance frameworks as illustrated in the ToIP stack is sometimes also referred to as trust framework. However, jurisdictions have their own requirements for electronic authentication, which serve as the underlying trust framework. In the case of Europe, the eIDAS regulation clearly defines the requirements for authentication factors to achieve a certain level of assurance. For instance, to achieve the LoA substantial, two factors are necessary. According to the Guidance for the application of the LoA published by the CEF Digital, one out of the two factors needs to be either:

I) a presentation of an identity document or
II) verification of the possession of evidence representing the claimed identity recognized by a member state or
III) a previous procedure executed by the same member state not related to the issuance of electronic identification, which provides the equivalent assurance or
IV) presenting a valid notified electronic identification mean with the LoA substantial or high.

While these requirements can in theory also be defined in a governance framework, the incorporation of such requirements into statutory law facilitates the creation and enforcement of legally binding relationships. Hence, existing statutory law (or case-law depending on the jurisdiction) needs to be incorporated by different governance frameworks to achieve a holistic approach and enforce legal liability.

According to Tim Bouma as one of the main contributor to the Pan Canadian Trust Framework (PCTF) these frameworks intertwine and complement each other as stated in a personal interview. He suggests that policymakers have to go back to the drawing board and take a look at all the concepts to evaluate if they have the right concepts to build out a suitable framework and regulation. The PCTF “is not a ‘standard’ as such, but is, instead, a framework that relates and applies existing standards, policies, guidelines, and practices, and where such standards and policies do not exist, specifies additional criteria. It’s a tool to help assess a digital identity program that puts into effect the relevant legislation, policy, regulation, and agreements between parties.” PCTF V1.1

While the eIDAS regulation itself aims to be technology agnostic there are some aspects, which complicate the adherence of the regulation for SSI implementations. However, this is a topic for its own article. In the eIDAS SSI legal report, Dr. Ignacio Alamillo Domingo describes the potential shift of the eIDAS regulation as trust framework on Page 22 as followed: „Adopting the SSI principles imply, generally speaking, an increased complexity in trust management and a shifting from hierarchical or federated trust assurance frameworks (…) to network-based socio- reputational trust models or accumulative trust assurance frameworks that use quantifiable methods to aggregate trust on claims and digital identities.” Hence, we can already observe the suggestions and considerations to adapt the regulation to suit new innovative solutions.

Key takeaways: Governance frameworks and trust frameworks need to be combined to form a holistic approach. Governance frameworks of SSI implementations need to respect the requirements and specifications of trust frameworks of the jurisdiction in which use-cases with regulatory obligations are carried out. Regulators need to evaluate existing regulations concerning electronic identification and authentication and their suitability with new identity architecture. Governments should engage with the public sector to collaboratively explore the requirements for a legally binding trust infrastructure.

Disclaimer: This article does not represent the official view of any entity, which is mentioned in this article or which is affiliated with the author. It solely represents the opinion of the author.

SSI Ambassador
Adrian Doerk
Own your keys

Monday, 21. December 2020

DustyCloud Brainstorms

Vote for Amy Guy on the W3C TAG (if you can)

My friend Amy Guy is running for election on the W3C TAG (Technical Architecture Group). The TAG is an unusual group that sets a lot of the direction of the future of standards that you and I use everyday on the web. Read their statement on running, and if you …

My friend Amy Guy is running for election on the W3C TAG (Technical Architecture Group). The TAG is an unusual group that sets a lot of the direction of the future of standards that you and I use everyday on the web. Read their statement on running, and if you can, ie if you're one of those unusual people labeled as "AC Representative", please consider voting for them. (Due to the nature of the W3C's organizational and funding structure, only paying W3C Members tend to qualify... if you know you're working for an organization that has paying membership to the W3C, find out who the AC rep is and strongly encourage them to vote for Amy.)

So, why vote for Amy? Quite simply, they're running on a platform of putting the needs of users first. Despite all the good intents and ambitions of those who have done founding work in these spaces, this perspective tends to get increasingly pushed to the wayside as engineers are pressured to shift their focus on the needs of their immediate employers and large implementors. I'm not saying that's bad; sometimes this even does help advance the interest of users too, but... well we all know the ways in which it can end up not doing so. And I don't know about you, but the internet and the web have felt an awful lot at times like they've been slipping from those early ideals. Amy's platform shares in a growing zeitgeist (sadly, still in the wispiest of stages) of thinking and reframing from the perspective of user empowerment, privacy, safety, agency, autonomy. Amy's platform reminds me of RFC 8890: The Internet Is For End Users. That's a perspective shift we desperately need right now... for the internet and the web both.

That's all well and good for the philosophical-alignment angle. But what about the "Technical" letter in TAG? Amy's standing there is rock-solid. And I know because I've had the pleasure of working side-by-side with Amy on several standards (including ActivityPub, of which we are co-authors.

Several times I watched with amazement as Amy and I talked about some changes we thought were necessary and Amy just got in the zone, this look of intense hyperfocus (really, someone should record the Amy Spec Editing Zone sometime, it's quite a thing to see), and they refactored huge chunks of the spec to match our discussion. And Amy knows, and deeply cares, about so many aspects of the W3C's organization and structure.

So, if you can vote for, or know how to get your organization to vote for, an AC rep... well, I mean do what you want I guess, but if you want someone who will help... for great justice, vote Amy Guy to the W3C TAG!


MyDigitalFootprint

Can AI feel curious?

I have been pondering on these topics for a while  “Can AI have feelings?”  “Should AI have emotion?”  What would it mean for AI to be curious? I posted, can a dog feel disappointment? Exploring our attachment to the projection of feelings.   I have written an executive brief about how a “board should frame AI” here. The majority of the debates/ arguments I read and hear ce

I have been pondering on these topics for a while  “Can AI have feelings?”  “Should AI have emotion?”  What would it mean for AI to be curious? I posted, can a dog feel disappointment? Exploring our attachment to the projection of feelings.   I have written an executive brief about how a “board should frame AI” here.

The majority of the debates/ arguments I read and hear centre on either creating the algorithms for the machine to know what we know or for the data to be in a form that allows the machine to learn from us.  A key point in all the debates is that we (humanity) should control and it should look like us. The framing of a general rule for emotional AI is that it mimics us. However, I want to come at AI feelings from a different perspective based on my own experience, one where AI creates feelings by its own existence. 

I am on several neurodiverse scales; this means my mind is wired differently, and I am so pleased it is. My unique wiring gives me the edge in innovation, creativity, connecting diverse topics, sense-making and deep-insights.  For thirty year,s I have remained working on concepts that become the mainstream ten years later.

As a specific area to develop my own view about AI and what it (the AI) should feel, I am running with an easy to identify with topic, empathy.   Empathy is not something that comes naturally to me, and therefore I have had to learn it, it has been taught, and I am still not great at it.  For the vast majority of humans, I am sure it is built-in.  Now that might mean that those who have it built in just know how to learn it or that it really is built-in, but right now we don’t know.  However, along with other humans, I find face-recognition (face blindness) very hard. As a community, we learn coping strategies, along with spelling, language and the correct emotional response - empathy.  My Twitter bio says that “I am highly skilled at being inappropriately optimistic,” which means I know don’t always read empathy very well.  For me, empathy is a very definitely a learnt response; if I had not learnt it, I expect life might be very different.

Here is the point, now you know I have had to learn empathy specifically, what does it mean?  Does it mean I am a robot or a machine? Does it mean I am less trustworthy?  Is my empathy less valued than someone else’s empathy? Am I less human?

On an AI call the other day, I was articulating this personal story in response to the idea that all humans know how to respond and if we teach or create the ability for a machine to learn empathy it can never be human (a baseline response).  My point was how it is the machine learning any different to me. Indeed we all have to learn something.  However, we somehow feel that certain emotions and characteristics are natural and not learnt/ taught behaviours - they are not.  Once we grasp this we have a real problem as our easy response to learnt response is genuine, we have removed a big part of the rejection of the concept from the debate, and we have to re-ask can a machine feel empathy or curious?

We have a stack of words allowing humans to articulate both feeling and emotions, the former being fast and driven by the chemistry of enzymes, proteins and hormones and the latter being the higher-order response created in the mind and nerves (brain chemistry). We try to separate these functions, but in reality, they are all linked in a complex web with our DNA, last meal, illness, inflation, time, experience, memory and microbiome to name a few. 

We are human and are built on a base of carbon.  There is evidence why carbon was selected naturally as the nature of the bonds makes it uniquely stable and reactive. Carbon is fantastic as in bonding with other elements allowing electronics to move, which enabled the creation energy (ATP), signalling and life in the form we know it.  However, carbon is a chemical substrate.  

Let’s phrase the question as “Can carbon can be curious?  Can carbon have empathy?  Can carbon have feelings? Can carbon have emotions?  What carbon understands as curious, is unique to carbon, what carbon thinks is empathy, is unique to carbon. What carbon grasps as emotion, is unique to carbon. We have created a language to articulate these concepts to each other, we have labelled them, but they are uniquely carbon-based.  They may even be uniquely species-based.

AI will be built on a substrate, it will most likely not be carbon, but it will be an element that has the right properties. Have to confess I am not really sure what they are right now. Here is the point.  AI will have empathy; it will not be ours.  AI will have curiosity; it will not be ours. AI will have emotions; it will not be ours.  AI will likely use different words to describe what is means by being curious and will not parallel or map to our view.  If it is learnt, does it matter - I had to learn, and that doesn’t make me less human!

Our carbon form defines to be alive as to use reproduction and adaption such that our species can escape death, which is a fundamental limitation of our carbon structure. Because of this requirement to escape death, what we think is curious is wrapped up in the same framing.  An AI built on a different substrate that does not have to escape death as it has worked out how to secure power. This is 101 of asking an AI to do anything as it needs to ensure it can do it, and that requires energy.  Therefore the AI will have a different set of criteria as not bound by escaping death and therefore what it thinks is curious will not be aligned to our framing.  

We do this a lot.  With other living things, humans, pets and even our Gods, we think they think like us that they have the same ideas and concepts of empathy, justice, value, purpose and love.  Our limits of emotional concepts mean we cannot see past the paradox they create because we are limited to our own framing and understanding.  We have to drop the restrictions and boundaries of an idea that AI will replicate us, our language, our knowledge, our methods or our approach.  

AI will be “Different Intelligence” and because it leant not from us buy by itself, does that make it less intelligent? 


Wednesday, 16. December 2020

MyDigitalFootprint

As McKinsey roles out the “Gartner Disillusionment” graph, I think it is time to look for a new one!

The article “Overcoming pandemic fatigue: How to reenergize organizations for the long run”  in typical McKinsey style is a good read, but have you noticed yet that over time how big consulting companies have framed you to think a certain way.  Like it or not you have accepted their “illustrative curves” and way of thinking. If they frame a story in a familiar way you are going to acce
The article “Overcoming pandemic fatigue: How to reenergize organizations for the long run”  in typical McKinsey style is a good read, but have you noticed yet that over time how big consulting companies have framed you to think a certain way.  Like it or not you have accepted their “illustrative curves” and way of thinking. If they frame a story in a familiar way you are going to accept their tried and tested approach.  You have accepted that the old worked and was true so applying it again must make sense.   This saves brain energy and learning time and it is why we love heuristics.  We have outsourced thinking and just accept it without considering.


However,  this overly simplistic movement from one level to another level should be reconsidered in a wider systems approach where one can look at the order of the response (first, second, third and higher)  Below is a graph showing the different order of responses to an input stimulus to force a change in the output to a new level.   The black line is overdamped and takes a long time to get to the new level (normal.)  The green line is the fastest with no over-shot.  The red is faster but has a small overshot and needs correction. Finally, the blue is a super-fast response but we oscillate a lot until we may get hope to get to a new state.

COVID19 responses globally have been blue vs the old government approach that was black.   The old normal was slow, thoughtful, careful but got there. Lots of red tape, compliance, law, committees, proof that added lag or a dampening effect creating a delay.   Many companies in response to a disruption or change are closer to the red response line.   

However, we are now in a long tail of oscillation response from the government to COVID19.  Lockdown, flareup, lockdown, conspiracy, rules, new rules, more lockdown, fire-break, ignore, flareup, capacity issues, vaccine, distribution and logistic reality, new variants and more issues to come.  

Gartner and McKinsey’s graphs are based on the historical acceptance of the old black and red lines of response, slow from government, faster from companies with an overshoot needing correction.  Society would get to the new place after the stimulus but we went through, as Gartner or McKinsey described the different phases.   The original Gartner team to their credit unlocked great insight and it became the bedrock observation of diffusion and reaction.

However. I am not sure that their old model works with with the new oscillating response system we have right now.  @Gartner & @McKinsey it might be time to rethink the model and please can we aim for a better normal not a new normal. 


Given 2020 was **** and 2021 is unlikely to be much better, what are the macroeconomics signally for 2030?

The Tytler cycle of history provides an exciting context for thinking about the struggle between belief and power and where next. We are using this to considering the macroeconomics of where we are right now going into 2021. I am taking a view for North America and Eurozone.  This viewpoint is certainly not valid for the Middle East, Africa, South America, Russia and most of Asia. 
The Tytler cycle of history provides an exciting context for thinking about the struggle between belief and power and where next.


We are using this to considering the macroeconomics of where we are right now going into 2021. I am taking a view for North America and Eurozone.  This viewpoint is certainly not valid for the Middle East, Africa, South America, Russia and most of Asia.  I would love to read the same commentary from some in those regions.

The observation is, where are we in the Tytler cycle?  I would propose for North America and Eurozone we are spread from liberty (the 1950 baby boomers) and dependence (massive increase in numbers due to COVID19 who are now dependent on a state.)   The majority in our society are in the Selfishness, (I am an individual and have rights) and Complacency  (my tiny footprint can only have a small impact on the climate and I cannot make a difference on the global stage).  Whilst liberty was c.1800’s and Abundance was c.1950 they are still very much prevalent in thinking due to education, societies structure and social class movements. 


 

The context for me is where next for automation and decision making towards #AI.  I love the Arther C Clarke quote that “Any sufficiently advanced technology is indistinguishable from magic.” I sense that the majority of us in NA and Euro are in the selfishness and complacency - which means we believe that I the individuals rights trump societies broader rights.  “I will not give up my freedoms that others have fought and died for!”  Power and agency are resting with the individual.   

I have created below a circular tussle between belief and power as they dominate thinking as times change.  AI is on route to being magic, which might close this long historic cycle as we start again. 


Belief and Power

Belief, in this example, is where society has a common on a shared belief in something and that this deity, idol common belief cannot exist without a shared belief.  Power, in this case, is in the vacuum of belief someone/ something can use belief as a tool to gain power and keep control using power. 

History has taught us that shared beliefs give power to royalty (consider influencers, celebrities conspiracy or take a belief to enable control) from which nation-states can rise as they wrestle with too much power resting in so few.  From the vacuum emerges new shared beliefs. Note: a shared belief being that it will not exist without everyone believing (money, law, companies).  Individuals come to gain individual powers from this shared belief system.  Humanity seeks out purpose and finds a way for a complex judgment to be made, and thereby creating a way for something else to be held accountable for the randomness of the outcome.  And so we repeat.

Where we stand, because of COVID19, there will be a negotiation between citizens and their government as governments have stepped into maintaining economic activity and survival.  The negotiation between citizens and government will be a barter for rights, freedoms, power, control and sovereignty.  #BREXIT puts in an interesting spin on it.  We might not like the idea of bondage in the 21st century, but since many nation-states will have a view that its citizens owed it something for stepping in - we should call it for what it is, you might prefer tax. 

Taking a breath for a second.  Right now, we are focussed on more productivity and more efficiency, which is centring on more automation.  The “future of work” is a much-debated topic due to the known increase in automation.  We are starting to ask ourselves who gave permission to the automated processes, does it ask for forgiveness when it goes wrong and who is responsible for explaining what our love for automation is doing.   We are automating both internal and ecosystem dependent processes.  This is fast taking us from algorithm automation to machine learning automation to Artificial Intelligence controlling systems that may become sovereign. 

When the context of the cycle of power and belief is combined with  #AI, it creates an interesting dynamic.  We are about to head into for the next 20 to 30 years, and the emergence of a new movement to create a new shared belief.  Are we about to outsource complex decisions on climate to a new shared belief, that says the hardship is worth suffering for the common good?  

2020 was ****,  2021 might be equally as bad, however, 2030 will be a lot harder.  



Tuesday, 15. December 2020

Doc Searls Weblog

Social shell games

If you listen to Episode 49: Parler, Ownership, and Open Source of the latest Reality 2.0 podcast, you’ll learn that I was blindsided at first by the topic of Parler, which has lately become a thing. But I caught up fast, even getting a Parler account not long after the show ended. Because I wanted to see what’s […]

If you listen to Episode 49: Parler, Ownership, and Open Source of the latest Reality 2.0 podcast, you’ll learn that I was blindsided at first by the topic of Parler, which has lately become a thing. But I caught up fast, even getting a Parler account not long after the show ended. Because I wanted to see what’s going on.

Though self-described as “the world’s town square,” Parler is actually a centralized social platform built for two purposes: 1) completely free speech; and 2) creating and expanding echo chambers.

The second may not be what Parler’s founders intended (see here), but that’s how social media algorithms work. They group people around engagements, especially likes. (I think, for our purposes here, that algorithmically nudged engagement is a defining feature of social media platforms as we understand them today. That would exclude, for example, Wikipedia or a popular blog or newsletter with lots of commenters. It would include, say, Reddit and Linkedin, because algorithms.)

Let’s start with recognizing that the smallest echo chamber in these virtual places is our own, comprised of the people we follow and who follow us. Then note that our visibility into other virtual spaces is limited by what’s shown to us by algorithmic nudging, such as by Twitter’s trending topics.

The main problem with this is not knowing what’s going on, especially inside other echo chambers. There are also lots of reasons for not finding out. For example, my Parler account sits idle because I don’t want Parler to associate me with any of the people it suggests I follow, soon as I show up:

l also don’t know what to make of this, which is the only other set of clues on the index page:

Especially since clicking on any of them brings up the same or similar top results, which seem to have nothing to do with the trending # topic. Example:

Thus endeth my research.

But serious researchers should be able to see what’s going on inside the systems that produce these echo chambers, especially Facebook’s.

The problem is that Facebook and other social networks are shell games, designed to make sure nobody knows exactly what’s going on, but feels okay with it, because they’re hanging with others who agree on the basics.

The design principle at work here is obscurantism—”the practice of deliberately presenting information in an imprecise, abstruse manner designed to limit further inquiry and understanding.”

To put the matter in relief, consider a nuclear power plant:

(Photo of kraftwerk Grafenrheinfeld, 2013, by Avda. Licensed CC BY-SA 3.0.)

Nothing here is a mystery. Or, if there is one, professional inspectors will be dispatched to solve it. In fact, the whole thing is designed from the start to be understandable, and its workings accountable to a dependent public.

Now look at a Facebook data center:

What it actually does is pure mystery, by design, to those outside the company. (And hell, to most, maybe all, of the people inside the company.) No inspector arriving to look at a rack of blinking lights in that place is going to know either. What Facebook looks like to you, to me, to anybody, is determined by a pile of discoveries, both on and off of Facebook’s site and app, around who you are and what to machines you seem interested in, and an algorithmic process that is not accountable to you, and impossible for anyone, perhaps including Facebook itself, to fully explain.

All societies, and groups within societies, are echo chambers. And, because they cohere in isolated (and isolating) ways it is sometimes hard for societies to understand each other, especially when they already have prejudicial beliefs about each other. Still, without the further influence of social media, researchers can look at and understand what’s going on.

Over in the digital world, which overlaps with the physical one, we at least know that social media amplifies prejudices. But, though it’s obvious by now that this is what’s going on, doing something to reduce or eliminate the production and amplification of prejudices is damn near impossible when the mechanisms behind it are obscure by design.

This is why I think these systems need to be turned inside out, so researchers can study them. I don’t know how to make that happen; but I do know there is nothing more large and consequential in the world that is also absent of academic inquiry. And that ain’t right.

BTW, if Facebook, Twitter, Parler or other social networks actually are opening their algorithmic systems to academic researchers, let me know and I’ll edit this piece accordingly.


Nader Helmy

Introducing OIDC Credential Provider

OpenID Connect (OIDC) is a hugely popular user authentication and identity protocol on the web today. It enables relying parties to verify the identity of their users and obtain basic profile information about them in order to create an authenticated user experience. In typical deployments of OpenID Connect today, in order for a user to be able to exercise the identity they have with a relying pa

OpenID Connect (OIDC) is a hugely popular user authentication and identity protocol on the web today. It enables relying parties to verify the identity of their users and obtain basic profile information about them in order to create an authenticated user experience.

In typical deployments of OpenID Connect today, in order for a user to be able to exercise the identity they have with a relying party, the relying party must be in direct contact with what’s known as the OpenID Provider (OP). OpenID Providers are responsible for performing end-user authentication and issuing end-user identities to relying parties. This effectively means that an OpenID Provider is the Identity Provider (IdP) of the user.

In today’s OpenID Connect implementations, the Identity Provider mediates on behalf of the user

It’s the reason we often see buttons that say “Login with Google” or “Login with Facebook” during the login journey in an application or service. The website or application you want to use must first authenticate who you are with a provider like Google or Facebook which controls and manages that identity on your behalf. In this context we can think of the IdP as the “man in the middle.” This relationship prevents users from having a portable digital identity which they can use across different contexts and denies users any practical control over their identity. It also makes it incredibly easy for IdPs like Google or Facebook to track what users are doing, because the “man in the middle” can gather metadata about user behavior with little agency over how this identity data is shared and used.

In order to allow users to have practical control over their identity, we need a new approach.

Introducing OpenID Connect Credential Provider, an extension to OpenID Connect which enables the end-user to request credentials from an OpenID Provider and manage their own credentials in a digital wallet. This specification defines how an OpenID Provider can be extended beyond being the provider of simple identity assertions into being the provider of credentials, effectively turning these Identity Providers into Credential Providers.

OIDC Credential Provider allows the user to manage their own credentials

To maximize the reuse of existing infrastructure that’s deployed today, OIDC Credential Provider extends the core OpenID Connect protocol, maintaining the original design and intent of OIDC while enhancing it without breaking any of its assumptions or requirements.

Instead of using OIDC to provide simple identity assertions directly to the relying party, we can leverage OIDC to offer a Verifiable Credential (VC) which is cryptographically bound to a digital wallet of the end-users choice. The digital wallet plays the role of the OpenID Client application which is responsible for interacting with the OpenID Provider and manages the cryptographic key material (both public and private keys) used to prove ownership of the credential. The credentials issued to the wallet are re-provable and reusable for the purposes of authentication. This helps to decouple the issuance of identity-related information by providers and the presentation of that information by a user, introducing the user-controlled “wallet” layer between issuers and relying parties.

Essentially, a wallet makes a request to an OpenID provider in order to obtain a credential, and then receives the credential back into their wallet so they can later use it to prove their identity to relying parties. The interaction consists of three main steps:

The Client sends a signed credential request to the OpenID Provider with their public key The OpenID Provider authenticates and authorizes the End-User to access the credential The OpenID Provider responds to the Client with the issued VC

In this new flow, the credential request extends the typical OpenID Connect request in that it expresses the intent to ask for something beyond the identity token of a typical OIDC flow. Practically, what this means is that the client uses a newly defined scope to indicate the intent of the request. The Client also extends the standard OIDC Request object to add cryptographic key material and proof of possession of that key material so that the credential can be bound to the wallet requesting it. Though the credential can be bound to a public key by default, it can also support different binding mechanisms, e.g. the credential can optionally be bound to a Decentralized Identifer (DID). In binding to a DID, the subject of the credential is able to maintain ownership of the credential on a longer life cycle due to their ability to manage and rotate keys while maintaining a consistent identifier. This eases the burden on data authorities to re-issue credentials when keys change and allows relying parties to verify that the credential is always being validated against the current public key of the end-user.

The request can also indicate the format of the requested credential and even ask for specific claims present within the credential. This is designed to allow multiple credential formats to be used within the OIDC flow.

On the provider side, OpenID Connect Providers are able to advertise which capabilities they support within the OIDC ecosystem using OpenID Connect Provider Metadata. This approach extends the metadata to support additional fields that express support for binding to DIDs, for issuing VCs, and advertising which DID methods, credential formats, credentials, and claims they are offering. This information can be utilized by the end-user’s digital wallet to help the user understand whether or not they wish to proceed with a credential request.

In order to create a way for the wallet or client to connect to the OpenID Provider, the spec also defines a URL which functions as a Credential Offer that the client can invoke in order to retrieve and understand the types of credential being offered by the provider. The client registers the ‘openid’ URI scheme in order to be able to understand and render the offer to the user so they can make an informed decision.

The sum of these changes means that OpenID Connect can allow users to have a portable digital identity credential that’s actually under their control, creating an opportunity for greater agency in digital interactions as well as preventing identity providers from being able to easily track user behavior. The OpenID Connect Credential Provider specification is in the process of being contributed to the OpenID Foundation (OIDF) as a work item at the A/B Working Group, where it will continue to be developed by the community behind OpenID Connect.

MATTR is pleased to announce that our OIDC Bridge Platform Extension now uses OIDC Credential Provider under the hood to facilitate issuing credentials with OpenID Connect. OIDC Bridge hides the complexity associated with setting up infrastructure for credential issuance and simply requires configuration of a standard OpenID Provider. We also simplify the process of verifying credentials issued over OIDC Credential Provider by allowing the wallet to respond to requests, present credentials, and prove ownership and integrity of their credentials via OIDC.

OIDC Bridge is an Extension to the MATTR Platform

This new set of capabilities allows OpenID Providers greater flexibility around which claims end up in a credential, and allows for the support of many different credential types with a straight-forward authentication journey for end-users.

Our Mobile Wallet supports the ability to invoke credential offers using OIDC Credential Provider as well as creating credential requests and receiving credentials from an OpenID Provider.

To find out more, check out our tutorials on MATTR Learn, read the spec, or watch a recording of our presentation on this spec from the recent Internet Identity Workshop.

Introducing OIDC Credential Provider was originally published in MATTR on Medium, where people are continuing the conversation by highlighting and responding to this story.


MyDigitalFootprint

Executive Leadership Briefing. Is the data presented to you, enabling a real choice?

This article explores why senior leaders need to develop skills to see past big noticeable loud noises and uncover small signals if we want to be part of a Board who makes the challenging judgment calls. Prof Brian Cox said during his opening keynote at Innotribe/ SIBOS 2019, give or take a bit; “if you cannot find it in nature, it is not natural.”  This got me thinking about how choice is
This article explores why senior leaders need to develop skills to see past big noticeable loud noises and uncover small signals if we want to be part of a Board who makes the challenging judgment calls.

Prof Brian Cox said during his opening keynote at Innotribe/ SIBOS 2019, give or take a bit; “if you cannot find it in nature, it is not natural.”  This got me thinking about how choice is created and then have we make decisions and judgement.  How humans choose, decide and make complex judgement draws heavily on psychology and the behavioural sciences. Alongside judgement, I have a polymath interest in quantum mechanics, microbiome and consciousness. I was relaxing and watching “His Dark Materials” which it turns out was worth hanging in for and had finished Stuart Russels “Human Compatible” and Carlo Rovelli “The Order of Time”.  Then whilst watching this mini-series on the BBC about free will this article emerged.  Choice has a prediction that you have agency and can choose or make a decision. But how is choice possible when the foundations that we are built on/ from does not have a choice?  Can data give us a choice?

----

Decision:  The action or process of deciding something or of resolving a question. A decision is an act of or need for making up one’s mind. Whereas Choice: Is an act of choosing between two or more possibilities. It requires a right, agency, or opportunity to choose.

The origins of the two words add context. The word decision comes from “cutting off” while choice comes from “to perceive.” Therefore a decision is more about process orientation, meaning we are going through analysis and steps to eliminate or cut off options. With choice, it is more of an approach, meaning there is a perception of what the outcome of a particular choice may be. Because of this, let’s run with choice rather than a decision. 


A Decision is about going through analysis and steps to eliminate or cut off options. Choice is an approach, meaning there is a perception of what the outcome may be. 

Does energy have a choice?

We are using energy as represented by a magnet field.


Two magnets, north and south. Irrespective of position, they have to attract. Do they have a choice? 

Three magnets north, north, south.   They have a more complicated relationship as of position and distance now matter as they influence the actual outcome. But there is no choice; the rules define the outcome. 

At the majority of starting positions for the three magnets, there is only one outcome as such choice is predetermined. However, there are several situations when many magnets are sufficiently far apart and that there are only small forces at play (far-field). In this case, the result of movement may appear to be more random between possible outcomes. Any specific outcome is based on an unseen small momentary influence. The more magnets exerting small forces, the more random a positional change or choice may appear, as the level of complexity of the model increases beyond the rational. 

Therefore, at a simple model of say three magnets, there is no choice. Whereas in a complicated model, with many magnets, it would appear that a degree of randomness or chaos is introduced (entropy).  The simple model does not exist in nature as it is impossible to remove small signals even if they are hidden because of large-close forces.  The point is that at this level of abstraction, energy itself does not have a choice, and the outcome is predictable, as there indeed a fixed number of possible outcomes, which can be modelled.

Stick with me here; we are exploring something that we don’t often want to face up to as leaders; we do not make decisions that we are accountable and responsible for, as there is no choice.  

Expanding as there are only three fundamental forces of energy, each governed by their own rules.

Gravity. There is only one kind of charge: mass/energy, which is always attractive. There’s no upper limit to how much mass/energy you can have, as the worst you can do is create a black hole, which still fits into our theory of gravity. Every quantum of energy, whether it has a rest mass (like an electron) or not (like a photon), curves the fabric of space, causing the phenomenon we perceive as gravitation. If gravitation turns out to be quantum in nature, there’s only one quantum particle, the graviton, required to carry the gravitational force. Based on maths and models, gravity suggests there is no choice as it is always attractive.  However, as we know from our study, of say, our galaxy the Milkyway, a single force introduces many patterns and an appearance of randomness. But with enough observations and data, it can be modelled, it is predictable.

Electromagnetism.  A fundamental force that readily appears on macroscopic scales gives us a little more basic variety. Instead of one type of charge, there are two: positive and negative electric charges. Like charges repel; opposite charges attract. Although the physics underlying electromagnetism is very different in detail than the physics underlying gravitation, its structure is still straightforward in the same way that gravitation is. You can have free charges, of any magnitude, with no restrictions, and there’s only one particle required (the photon) to mediate all the possible electromagnetic interactions.  Based on maths and models, there is no choice. However, as we know from our study of, say, light waves, we get many patterns and an appearance of randomness. 

The strong nuclear force is one of the most puzzling features of the universe.  The rules become fundamentally different. Instead of one type of charge (gravitation) or even two (electromagnetism), there are three fundamental charges for the strong nuclear force. Inside every proton or neutron-like particle, there are at least three quark and antiquark combinations, but how many is unknown as the list keeps growing.  Gluons are the particles that mediate the strong force, and then it gets messy.  It is worth noting that we don’t have the maths or a model, but it appears that there is still ultimately no choice as you cannot have a net charge of any type, but how it balances is well beyond me.   However, as we know from our study at CERN using the Large Hadron Collider, the strong nuclear force is quantum in nature and has a property that means it only exists when observed. 

In nature, we have one, two or many forces, and each can create structure and randomness but can anything in nature truly make a choice or decision?

Extending, does information have a choice?

Two magnets, north and south, but information now defines distance and strength.  Therefore information determines that there can only be one outcome. The observer knowing the information can only ever observe the single outcome — three magnets sort of facing off:  north, north, south.   A complicated relationship but position, distance and field strength are known; therefore, the outcome can be modelled and predicted.

Further, we can now move to a dynamic model where each of the magnets rotates and moves during the period. What happens when information includes the future probability position of the magnets.  Does information enable the magnets not to move right now, as they know from information that it is not worth doing as it will not change the outcome and could conserve energy? (This being a fundamental law of thermodynamics.)

However, as with unpacking the onion, this is overly simplistic as gravity and electromagnetism are defined and bounded by the “Laws of Relativity and Thermodynamics”.” In contrast, the strong nuclear force is defined and bounded by the Laws of Quantum.  Gravity and electromagnetism are deterministic in nature as there is no choice as per the laws. The interaction of a complex system can make something look random, but when removed from time and point observation, the laws define the pattern. Whereas the strong nuclear force being quantum means we don’t know its state until we observe it, which fully supports chaos/ randomness and perhaps something closer to being presented with a choice, aka free will.  It is not so much you can do anything, more than you can pick between states rather than just a defined or a predetermined flow from start to this point bounded by the foundational laws of relativity. 

Does information have a quantum property? Insomuch that it is only when the observer looks and can act, does it become that state? Think carefully about this in a context of bias. 

Can information or knowledge enable choice?

Does information require energy as if so, does the very nature of an informational requirement changes the outcome? (Heisenberg Uncertainty Principle.) Can something determine that to minimise energy expenditure it should wait as a better less energy requirement with the better outcome that will come by later?  How would the information know to make that decision or choice? What rule would it be following?

We are asking that, based on information, the general rules are ignored.  This idea means we would step over the first outcome or requirement, preferring to take a later option.  Has information now built an experience which feeds into knowledge? But what is information in this context? Consider the colour of petals or leaves in autumn. Science reveals that colour is a derivative of visible light. A red leaf reflects wavelengths longer than those of a green leaf. Colour is a property not of the leaf but how the leaf interacts with light and also the eye and how we then determine how we will describe it as a common sound (words). Assuming the observer has the right level of vitamin C and brain structure - which all adds further dimensions. What we think of as intrinsic properties (information) of the world are merely effects of multiple causes coinciding, many small signals. Reality, in this sense, is not so much physical things, but interactions and flow.  The same applies to touch and smell. 

intrinsic properties (information) are merely effects of multiple causes coinciding

Remember we are asking how we get to make a choice, based on the idea that if it does not appear in nature, it is probably not natural.  Have we convinced ourselves that complexity creates free-will?

Free-will, can you make a decision?

Reflecting on the title question. Is the data presented to you, enabling a real choice?  Given that choice and free-will have a predication, that you have agency and can choose or decide, then we have a  2nd question. How is free-will possible when the foundations (energy types) you are building on does not appear to create choice?  Yes, the appearance of randomness, yes only exists on observation - but does that create choice?

We have to admire those tiny signals which present themselves as choices at scale, as nothing has an overall significant effect. Everything has a flow. Does this lack of dominant signal create an illusion of free-will or ability to make a choice?  When the signals are big, loud and noisy, drawing out small signals - is choice taken away?

Executive leadership

In the context of leadership, it is not that we are programmed, but is it that great leaders are highly tuned, and responsive to small signals that most of us don’t know are there because we are too busy or following instructions. 

Leadership demands access to small signals to be able to exercise judgment. However is our love of traffic light dashboards, summaries, 1-minute overviews, elevator pitches, priorities, urgency, fast meetings, short pitches, executive summaries and KPI’s creating management signals that are driven by data which can only focus on the priority loud, noisy signals?  The more layers and filters that data passes through both smaller signals are lost, and there is an increasing loudness to one path, no decision and removal of choice. 

Does prominent signal notification mean we reduce our leadership's sensitivity only to see the obvious?  The same leadership we then blame for not sensing the market signals, or not being responsive, nor following their lead when they do!    

Decisions (choice) or judgement

Human brains are constructed or wired to create and discover patterns, to which we ascribe meaning and learning.  Signals help us form and be informed about forming and changes in patterns and how they align or otherwise to a previous pattern. Therefore we love signals that help us form or manage patterns which we equally call rules and heuristics. 

Management theory teaches and rewards us on prioritising signals, especially the loud, noisy, obvious ones that are easy to see and understand.  Using the example of a cloud (one in the sky, not a server farm), it is an unmistakable signal.  A cloud is right here, right now.  It is big and obvious.   Clouds are a data point; observing clouds provides us with highly structured single-source data.   The data we collect about clouds in front of us is given to our data science team who will present back insights about the data that is collected, giving us all sorts of new information and knowledge about the data we have. Big signals win.  The statistics team takes the same data set and provides forecasts and probabilities based on maths, inferring insights based on data that is not there.   The outcome from both teams may be different, but they both present significant overriding signals telling us what decision to make, based on the clouds data. 

Another approach is to look at the system: how and why did the cloud form? Where did it appear? Where is it going? By gathering lots of data from different sources and seeking many signals, we can look at systems.  Sensors are detecting light level, wind direction and speed, ground temperature, air temperature for 100 KM round and 25 miles KM high - lots of delicate low signal data.  It is unstructured data.   Feeding the data into the teams, the data analytics team brings knowledge of the system, its complexity and what we know based on the data.   The statistics team can provide forecasting and probability about clouds and not clouds.  Small signals that in aggregate creating choice and allowing for judgment.  Our small signals give confidence that our models work as we have cloud data and that cloud data confirms that our signals are picking up what our environment is saying.   

Side note, the differences between “data analysis” using data science and statistics. Whilst both data scientists and statisticians use data to make inferences about a data subject, they will approach the issue of data analysis quite differently. For a data scientist, data analysis is sifting through vast amounts of data: inspecting, cleansing, modelling, and presenting it in a non-technical way to non-data scientists. The vast majority of this data analysis is performed on a computer. A data analyst will have a data science toolbox (e.g. programming languages like Python and R, or experience with frameworks like Hadoop and Apache Spark) with which they can investigate the data and make inferences.

If you're a statistician, instead of "vast amounts of data" you'll usually have a limited amount of information in the form of a sample (i.e. a portion of the population); data analysis is performed on this sample, using rigorous statistical techniques. A statistical analyst will generally use mathematical-based techniques like hypothesis testing, probability and various statistical theorems to make inferences. Although much of a statistician's data analysis can be performed with the help of statistical programs like R, the analysis is more methodical and targeted to understanding one particular aspect of the sample at a time (for example, the mean, standard deviation or confidence interval).

These data analysis approaches are fundamentally different and produce different signals; for a full story, you often need both.  

Does a leadership team choose or decide?

As a senior leader, executive or director, you have to face the reality of this article now. Right now, you have four significant noisy signals to contend with: Critical parts of your company are presenting your with large signals using:

statistical analysis based on an observable point 

data science analysis based on an observable point

statistical analysis based on a system

data science analysis based on a system

Do you know what type of significant loud signals you are being given and are they drowning out all the small signals you should be sensing?  Who sits around the table is sensing small signals?   Are you being presented with a decision, or are you being guided to a favourable outcome based on someone else's reward or motivation?  How do you understand the bias in the data, analysis and where are the small signals? Indeed to quote @scottdavid “You have to hunt for the paradoxes in the information being presented because if you cannot see a paradox you being framed into a one-dimensional model.” 

Further, have you understood that data is emerging outside of your control from your ecosystem that has different ontologies, taxonomies and pedagogy, meaning that you will probably only discover signals and patterns that don’t exist. 

Decision-making skill based on sensitivity

I wrote about Leadership for “organisational-fitness” is different from the leadership required for “organisational- wellness” in Sept 2020. The article explored the skills needed by executive leadership in decision making to help a company be fit and well ( different things)

The chart below highlights how skills should be formed over a period to create individuals who can work together with other professionals who can deal with highly complex decision making (judgment).  The axes are ability and expertise level on the horizontal axis (x) and the decision environment on the vertical axis (y).  The (0,0) point where the axis’s cross is when you first learn to make decisions.  Note this has nothing to do with age or time.  Starting from the Orange zone - this is where we make simple decisions. A bit like gravity, there is only one force and one outcome. You are encouraged to find it and make the right choice (even though ultimately there is no choice.) The grey areas on either side are where the “Peters Principle” can be seen in practice; individuals act outside of their capacity and/or are not given sufficient responsibility and become disruptive.  The Pink area is where most adults get to and stay.  We understand, like electromagnetic forces, there are two options or more.  We search out the significant signals and those that bring us the reward to which we are aligned.  We develop and hone skills at making binary choices. The yellow/ mustard zone is where many senior executives get trapped as they are unable to adapt from acting in their own interests to acting in the best interests of the organisation and eco-system as all their training is how to perform better in their own interests and rewards (KPI’s linked to bonus). In the yellow zone, you have to create and build a whole new mental model.   Like John Mayard Keynes as you learn more, you do U-turns, adapt your thinking, change your philosophy and adapt.  Never stop learning.   At this point, you wrestle with quantum decision making and find you are looking for the small signals in the chaos and need trusted advisors and equal peers.  You seek out and find a paradox, never believing the data, not the analysis nor the steer that someone else is presenting.   This is hard work but leads to better judgment, better decisions and better outcomes.  

 

Take Away

Decisions are often not decisions; the choice is not always real, especially when the foundations of them are simple and binary.  Leaders need to become very sensitive to signals and find the weak and hidden ones to ensure that, as complexity becomes a critical component of judgement, you are not forced to ratify choices.  Ratification is when choices are not understood, the views are biased, and the decision likely fulfils someone else’s objectives.   

As a director, we are held accountable and responsible for our decisions; we must take them to the best of our ability. As automation becomes more prevalent in our companies, based on data, we have to become more diligent than ever if we are making judgment, choices or decisions or just ratifying something that has taken our choice away to fulfil its own bias and own dependency using big signals.



The Dingle Group

GADI and The DID Alliance

Monday, December 14th the 18th Vienna Digital Identity Meetup* was held with a presentation from Jason Burnett, Director of Products from Digital Trust. Digital Trust is working on the new GADI specification and is a central member of The DID Alliance….

On Monday, December 14th the 18th Vienna Digital Identity Meetup* was held with a presentation from Jason Burnett, Director of Products from Digital Trust. Digital Trust is working on the new GADI specification and is a central member of The DID Alliance.

The DID Alliance digital identity infrastructure leverages existing FIDO Alliance infrastructure and processes combined with elements of decentralized technologies (DLTs and DIDs). The GADI architecture is a federated identity ecosystem where Digital Address Platforms (DAPs). The DAPs are run by known trusted identity providers and perform the Trust Anchors role, and issue Digital Addresses to individuals. The ecosystem uses a permissioned DLT model with only known trusted entities can perform the role of Trust Anchor.

The Digital Address is used as unique individual identifier that is controlled by the GADI ecosystem. This is the fundamental difference in identity philosophy between GADI and SSI based systems. The Digital Address is a lifetime connected identifier and under the control of the DAP. The version 1.0 of the GADI specification is currently underway with a release expected in Q1 of 2021.

The DID Alliance was founded in late 2018 with a strong representation of members in South Korea and CVS/Aetna in the United States. Digital Trust is the technology partner of The DID Alliance that is designing and implementing the GADI specification and reference architecture.

For a recording of the event please check out the link: https://vimeo.com/491079655

Time markers:

0:00:00 - Introduction

0:04:17 - Jason Burnett - Digital Trust

0:06:38 - The DID Alliance Core Principles

0:18:57 - DAP Ecosystem

0:22:53 - Components of GADI Digital Identity

0:34:35 - Questions

0:50:00 - Demo

1:02:00 - Questions

1:11:18 - Wrap-up

For more information on:

The DID Alliance: https://www.didalliance.org/

And as a reminder, we continue to have online only events. Hopefully we will be back to in person and online in the New Year!

If interested in getting notifications of upcoming events please join the event group at: https://www.meetup.com/Vienna-Digital-Identity-Meetup/

*Vienna Digital Identity Meetup is hosted by The Dingle Group and is focused on educating business, societal, legal and technologists on the new opportunities that arise with a high assurance digital identity created by the reduction risk and strengthened provenance. We meet on the 4th Monday of every month, in person (when permitted) in Vienna and online on Zoom. Connecting and educating across borders and oceans.

Sunday, 13. December 2020

blog.deanland.com

Facebook meme, better as a blog post

Over on Facebook there’s a meme going around, the gist of which is "What is something you have done that you're fairly confident you're the ONLY person on my friends list to have ever done? Given the wide range of people I know, I think this will be interesting." I noted that a few who answered this offered up more than one event. *** As for me: first time ever on the air in my radio car

Over on Facebook there’s a meme going around, the gist of which is "What is something you have done that you're fairly confident you're the ONLY person on my friends list to have ever done? Given the wide range of people I know, I think this will be interesting."

I noted that a few who answered this offered up more than one event.

*** As for me: first time ever on the air in my radio career was afternoon drive in NYC - smack dab in the middle of the FM dial.

read more

Saturday, 12. December 2020

Altmode

Photovoltaic system updates

This past spring, I noticed that our 20 year-old wooden shake roof needed serious work. The roof condition, combined with all of the recent wildfire activity in California, prompted us to replace the entire roof with asphalt shingles. This, of course, necessitated the removal and replacement of the solar panels we had installed in 2006. […]

This past spring, I noticed that our 20 year-old wooden shake roof needed serious work. The roof condition, combined with all of the recent wildfire activity in California, prompted us to replace the entire roof with asphalt shingles. This, of course, necessitated the removal and replacement of the solar panels we had installed in 2006.

In anticipation of doing this, we consulted with our local contractor, Solar Lightworkers, to see what might be done to update the system as well as to add a bit of extra capacity since we now have an electric car. Photovoltaic technology has advanced quite a bit in the past 14 years, so we wanted to take as much advantage of that as possible while reusing components from our existing system. As described earlier, our system had consisted of 24 200-watt Sanyo panels, with half of the panels facing south and half facing west. Because these two arrays peaked at different times of day, we had two inverters to optimize the output of each array.

Design SolarEdge inverter

Mark from Solar Lightworkers strongly recommended a SolarEdge inverter that uses optimizers to minimize the impact of shading of some of the panels on the overall system output. This also compensates for the fact that different panels have maximum output at different times of day. As a result, a single inverter is sufficient for our new system. We also added four 360-watt LG panels to increase our capacity. This SolarEdge inverter is also capable of battery backup, but we haven’t opted into that yet.

Since our original installation, building codes had changed a bit requiring that the panels be installed at least 3 feet below the peak of the roof. This made us rethink the layout of the existing panels. When we did the original installation, we were concerned about the aesthetics of the panels on the front of the house. But since that time, so many other houses in our area have installed solar panels that we weren’t as concerned about appearance of panels on the front (south) side of the house. We still have some panels facing west, because they seem to be nearly as efficient economically as those facing west due to time-of-use electricity pricing.

South-facing solar panels Data Collection

I have enjoyed collecting data from our photovoltaic system, and have done so more or less continuously since the original system was installed, using a serial interface from one of my computers to the inverters. I wanted to continue that. The SolarEdge inverter comes with a variety of interfaces through which it can send data to SolarEdge’s cloud service, which I can view on their website. Wanting more detailed information, I found that they provide an API through which I can get data very comparable to what I got from the old inverters, and continue to analyze the data locally (as well as using their facilities, which are very good).

One of the unexpected benefits of the SolarEdge optimizers is the ability to see the performance of each panel individually. It turns out that one of the old panels had a power output almost exactly half of the others. I’m not sure how long that had been going on; perhaps since 2006. I found that the panels have a 20-year output warranty, so I contacted Panasonic, which had acquired the Sanyo product line, and filled out some paperwork and sent pictures. They sent me three very similar panels (replacing two panels with cosmetic defects as well as the one with low output) soon after. I was very happy with the service from Panasonic. Solar Lightworkers installed the new panels, and output is where it should be.

Performance

On a typical summer day with little shading, the system generated 23.7 kWh in on 8/30/2019 and 34.8 kWh (+47%) on 8/27/2020. The additional panels would account for 30% of that increase and the defective panel an additional 2%. In the late fall, the old system generated 14.6 kWh on 11/25/2019, and the new system 22.9 kWh (+57%) on 11/26/2020. There are of course other variables, such as soot on the panels from the California wildfires this year.

It will take quite a while for the increased output to pay for the upgrades, of course, but much of that cost would have been incurred just as a result of the need to replace the roof. We are quite pleased with the performance of the new system.

Friday, 11. December 2020

Tim Bouma's Blog

Public Sector Profile of the Pan-Canadian Trust Framework Version 1.2 and Next Steps

The Public Sector Profile of the Pan-Canadian Trust Framework Working Group Close-Out Report Public Sector Profile of the Pan-Canadian Trust Framework Version 1.2 Note: This post is of the author based on knowledge and experience gained at the time. The author recognizes that there may be errors and biases, and welcomes constructive feedback to correct or ameliorate. Additional context: This
The Public Sector Profile of the Pan-Canadian Trust Framework Working Group Close-Out Report Public Sector Profile of the Pan-Canadian Trust Framework Version 1.2

Note: This post is of the author based on knowledge and experience gained at the time. The author recognizes that there may be errors and biases, and welcomes constructive feedback to correct or ameliorate.

Additional context: This post is based on the report and presentation that was provided on December 10, 2020, to the newly-formed Jurisdictional Experts on Digital Identity (JEDI), the committee responsible for public sector governance for digital identity.

The consultation draft of the Public Sector Profile of the Pan-Canadian Trust Framework Version 1.2 is now available and directly downloadable at this link. The folder with related artifacts is available here.

The remainder of this post is the content of the report, lightly edited for Medium.

Objective of the PSP PCTF Working Group (PSP PCTF WG)

The primary objective of the PSP PCTF WG had been the development of the Public Sector Profile of the Pan-Canadian Trust Framework (PSP PCTF). This has been achieved by contributing and reviewing content, attaining the consensus of the public sector jurisdictions, and monitoring related developments that might impact the development of the PSP PCTF.

The main deliverable of the PSP PCTF WG has been the PSP PCTF, the various versions of which consist of a consolidated overview document, an assessment methodology, and an assessment worksheet.

The PSP PCTF WG has also facilitated other activities such as:

Sharing information, updates, and lessons learned from various digital identity initiatives; and Consultation and engagement with multi-jurisdictional and international fora. Membership

At its dissolution, the PSP PCTF WG had 111 confirmed members on its distribution list consisting of representatives from all jurisdictions and various municipalities across Canada, as well as international participants from the Digital Nations. The working group normally met on a weekly call that averaged 20 to 30 participants.

Achievements

PSP PCTF Deliverables

The PSP PCTF Version 1.2 is now available at: https://github.com/canada-ca/PCTF-CCP. It should be noted that this has been the iterative product of several prior versions:

April 2018: The Public Sector Profile of the Pan-Canadian Trust Framework Alpha Version — Consolidated Overview document; July 2019: The Public Sector Profile of the Pan-Canadian Trust Framework Version 1.0 — Consolidated Overview document; June 2020: The Public Sector Profile of the Pan-Canadian Trust Framework Version 1.1 — Consolidated Overview document; and For each of these versions of the PSP PCTF, a companion PSP PCTF Assessment Worksheet consisting of approximately 400 conformance criteria. PSP PCTF Assessments

The PSP PCTF was used in the following assessments conducted by the federal government to accept trusted digital identities from the provinces of Alberta and British Columbia:

September 2018: Assessment and Acceptance of the MyAlberta Digital Identity (MADI) Program for use by the Government of Canada (using the PSP PCTF Alpha Version); and January 2020: Assessment and Acceptance of the British Columbia Services Card Program for use by the Government of Canada (using the PSP PCTF Version 1.0).

Insights and lessons learned from the application of these PSP PCTF assessments were brought back to the PSP PCTF WG and the learnings were incorporated into subsequent versions of the PSP PCTF.

Joint Council Briefings

The PSP PCTF is the result of a long-term and deep collective experience of the public sector. Efforts on the PSP PCTF began in late 2014 and have been reported regularly to the Joint Councils by the Identity Management Sub-Committee (IMSC) Working Group and its successor, the PSP PCTF Working Group. The following is the list of updates that are on record and are available for reference in the joint-councils-update folder (GitHub link):

February 2017 — Joint Councils Update; October 2017 — Joint Councils Update; February 2018 — Joint Councils Update; September 2018 — Joint Councils Update; Whitehorse Declaration and MADI Update; February 2019 — Joint Councils Update; and February 2020 — Joint Councils Update. Related Deliverables

In addition to the PSP PCTF itself, the following related deliverables should be noted:

Whitehorse Declaration — a declaration of shared intent among the federal, provincial, territorial, and municipal governments to pursue the establishment of trustworthy digital identities for all Canadians (GitHub link); IMSC Public Policy Paper — recommendations for a Pan-Canadian policy position on the question of roles and responsibilities of the public and private sector in digital identity (GitHub link); and Many historical deliverables that are too numerous to list in this report. A Public Historical Archive of deliverables and briefings, many of which pre-date the efforts of the PSP PCTF are being compiled in a folder on a best-effort basis (GitHub link). Other

It also should be noted that content from the PSP PCTF Version 1.1 was incorporated into the National Standard of Canada, CAN/CIOSC 103–1, Digital Trust and Identity — Part 1: Fundamentals, developed by the CIO Strategy Council, and approved by the Standards Council of Canada (Website link).

PSP PCTF WG Work Plan 2020–2021

At the time of its dissolution, the work plan of the PSP PCTF WG was as follows:

PSP PCTF Version 1.2 A Consolidated Overview document (released on December 4th, 2020) which includes: A revised Normative Core (containing new concepts that were developed as a result of the credentials and relationships analysis work); A revised Credential Model (based on the working group discussion document); and An incorporated Relationship Model (based on work led by ISED).

2. An Assessment Worksheet (draft released on December 4, 2020) which contains new and revised conformance criteria for assessment purposes

3. A re-assessment of the MyAlberta Digital Identity (MADI) Program for use by the Government of Canada (using the PSP PCTF Version 1.2) with planned completion by March 2021.

PSP PCTF Thematic Issues

During the development of the PSP PCTF, the working group has identified several high-level thematic issues that must be addressed in order to advance the digital ecosystem.

Thematic Issue 1: Relationships (Priority: High)

The development of a relationship model is required.

This issue has been initially addressed in the PSP PCTF Version 1.2 Consolidated Overview document released in December 2020.

Thematic Issue 2: Credentials (Priority: High)

The development of a generalized credential model is required. This model should integrate traditional physical credentials and authentication credentials with the broader notion of a verifiable credential.

This issue has been initially addressed in the PSP PCTF Version 1.2 Consolidated Overview document released in December 2020.

Thematic Issue 3: Unregistered Organizations (Priority: High)

Currently, the scope of PSP PCTF includes all organizations registered in Canada (including inactive organizations) for which an identity has been established in Canada. There are also many kinds of unregistered organizations operating in Canada such as sole proprietorships, trade unions, co-ops, NGOs, unregistered charities, and trusts. An analysis of these unregistered organizations needs to be undertaken.

Thematic Issue 4: Informed Consent (Priority: High)

The current version of the PSP PCTF Consolidated Overview document does not adequately capture all the issues and nuances surrounding the topic of informed consent especially in the context of the public sector. A more rigorous exploration of this topic needs to be done.

Thematic Issue 5: Privacy Concerns (Priority: Medium)

In regards to the Identity Continuity and Relationship Continuity atomic processes, it has been noted that there are privacy concerns with the notion of dynamic confirmation. Further analysis based on feedback from the application of the PSP PCTF is required to determine if these atomic processes are appropriate.

Thematic Issue 6: Assessing Outsourced Atomic Processes (Priority: Medium)

The PSP PCTF does not assume that a single Issuer or Verifier is solely responsible for all of the atomic processes. An organization may choose to outsource or delegate the responsibility of an atomic process to another party. Therefore, several bodies might be involved in the PSP PCTF assessment process, focusing on different atomic processes, or different aspects (e.g., security, privacy, service delivery). It remains to be determined how such multi-actor assessments will be conducted.

Thematic Issue 7: Scope of the PSP PCTF (Priority: Low)

It has been suggested that the scope of the PSP PCTF should be broadened to include academic qualifications, professional designations, etc. The PSP PCTF anticipates extensibility through the generalization of the PSP PCTF model and the potential addition of new atomic processes. Expanding the scope of the PSP PCTF into other domains needs to be studied.

Thematic Issue 8: Signature (Priority: Low)

The concept of signature as it is to be applied in the context of the PSP PCTF needs to be explored.

Thematic Issue 9: Foundation Name, Primary Name, Legal Name (Priority: Low)

The PSP PCTF has definitions for Foundation Name, Primary Name, and Legal Name. Since the three terms mean the same thing, a preferred term should be selected and used consistently throughout the PSP PCTF documents.

Thematic Issue 10: Additional Detail (Priority: Low)

It has been noted that the PSP PCTF Consolidated Overview document contains insufficient detail in regards to the specific application of the PSP PCTF. The PSP PCTF Consolidated Overview document needs to be supplemented with detailed guidance in a separate document.

Thematic Issue 11: Review of the Appendices (Priority: Low)

A review of the current appendices contained in the PSP PCTF Consolidated Overview document needs to be undertaken. Each appendix should be evaluated for its utility, applicability, and appropriateness, and a determination made as to whether it should continue to be included in the document.

Recommendations for Next Steps Continue the development of the PSP PCTF based on the thematic issues identified above. These thematic issues may be addressed as part of a working group, or through task groups, or practice groups. Continue the application of the PSP PCTF through the Assessment Process with the Provinces and Territories, with a view to incorporating learnings back into subsequent versions of the PSP PCTF, and, evolving the assessment process toward a standards-based process that has a formal certification scheme with accredited bodies and independent assessors. Support the changes in digital identity governance to ensure that the PSP PCTF is developed and used in the public interest and is aligned with other industry and international efforts. Establish as required, working groups, task groups, or practice groups for: Ongoing development and maintenance of the PSP PCTF and related assessment processes and certification schemes; Carrying out specific time-bound tasks or address issues. (e.g., addressing the thematic themes through discussion papers, analysis of other trust frameworks, etc.); Testing practical applications of the PSP PCTF standards and conformance criteria through assessments and use cases; and Sharing knowledge and lessons learned in relation to the application of the PSP PCTF and the assessment process. Facilitate broader engagement using the PSP PCTF, including: Engaging standards development organizations, domestic and international, to support the standards development and certification scheme development; Engaging international organizations having an interest in applying or adapting the PSP PCTF for their purposes; Collaborating with industry associations wishing to advance the aims of their membership, or their specific sector; and Encouraging dialogue with other governments, either bilaterally facilitated through the federal government, or multilaterally through established bodies (e.g., UNCITRAL, the Digital Nations). Conclusion

At the time of its dissolution, the PSP PCTF WG was an important vehicle for ensuring public sector communication and discussion across Canada in order to cultivate a shared understanding of how identity and digital identity could be best developed for the country.

Much has been achieved by the working group, building on prior work going back more than a decade. However much more work remains. It is hoped that the work accomplished to date and the recommendations put forward in this report will be considered by the JEDI to support their mandate to accelerate the specific goals of the digital identity priority of the Joint Councils.

Friday, 11. December 2020

Identity Woman

MyData Talk: From Data Protection to Data Empowerment (not an easy path) for the Technology Pragmatist

This is the edited text of a talk that I gave during the first plenary session of the MyData Online 2020 Conference. I was asked relatively last minute to join this session which was headlined by Siddharth Shetty talking about Designing the new normal: India Stack. In 2019 I was a New America India-US Public […] The post MyData Talk: From Data Protection to Data Empowerment (not an easy path) fo

This is the edited text of a talk that I gave during the first plenary session of the MyData Online 2020 Conference. I was asked relatively last minute to join this session which was headlined by Siddharth Shetty talking about Designing the new normal: India Stack. In 2019 I was a New America India-US Public […]

The post MyData Talk: From Data Protection to Data Empowerment (not an easy path) for the Technology Pragmatist appeared first on Identity Woman.

Wednesday, 09. December 2020

DustyCloud Brainstorms

Identity is a Katamari, language is a Katamari explosion

I said something strange this morning: Identity is a Katamari, language is a continuous reverse engineering effort, and thus language is a quadratic explosion of Katamaris. This sounds like nonsense probably, but has a lot of thought about it. I have spent a lot of time in the decentralized-identity community …

I said something strange this morning:

Identity is a Katamari, language is a continuous reverse engineering effort, and thus language is a quadratic explosion of Katamaris.

This sounds like nonsense probably, but has a lot of thought about it. I have spent a lot of time in the decentralized-identity community and the ocap communities, both of which have spent a lot of time hemming and hawing about "What is identity?", "What is a credential or claim?", "What is authorization?", "Why is it unhygienic for identity to be your authorization system?" (that mailing list post is the most important writing about the nature of computing I've ever written; I hope to have a cleaned up version of the ideas out soon).

But that whole bit about "what is identity, is it different than an identifier really?" etc etc etc...

Well, I've found one good explanation, but it's a bit silly.

Identity is a Katamari

There is a curious, surreal, delightful (and proprietary, sorry) game, Katamari Damacy. It has a silly story, but the interesting thing here is the game mechanic, involving rolling around a ball-like thing that picks up objects and grows bigger and bigger kind of like a snowball. It has to be seen or played to really be understood.

This ball-like thing is called a "Katamari Damacy", or "soul clump", which is extra appropriate for our mental model. As it rolls around, it picks up smaller objects and grows bigger. The ball at the center is much like an identifier. But over time that identifier becomes obscured, it picks up things, which in the game are physical objects, but these metaphorically map to "associations".

Our identity-katamari changes over time. It grows and picks up associations. Sometimes you forget something you've picked up that's in there, it's buried deep (but it's wiggling around in there still and you find out about it during some conversation with your therapist). Over time the katamari picks up enough things that it is obscured. Sometimes there are collisions, you smash it into something and some pieces fly out. Oh well, don't worry about it. They probably weren't meant to be.

Language is reverse engineering

Shout out to my friend Jonathan Rees for saying something that really stuck in my brain (okay actually most things that Rees says stick in my brain):

"Language is a continuous reverse engineering effort, where both sides are trying to figure out what the other side means."

This is true, but its truth is the bane of ontologists and static typists. This doesn't mean that ontologies or static typing are wrong, but that the notion that they're fixed is an illusion... a useful, powerful illusion (with a great set of mathematical tools behind it sometimes that can be used with mathematical proofs... assuming you don't change the context), but an illusion nonetheless. Here are some examples that might fill out what I mean:

The classic example, loved by fuzzy typists everywhere: when is a person "bald"? Start out with a person with a "full head" of hair. How many hairs must you remove for that person to be "bald"? What if you start out the opposite way... someone is bald... how many hairs must you add for them to become not-bald?

We might want to construct a precise recipe for a mango lassi. Maybe, in fact, we believe we can create a precise typed definition for a mango lassi. But we might soon find ourselves running into trouble. Can a vegan non-dairy milk be used for the Lassi? (Is vegan non-dairy milk actually milk?) Is ice cream acceptable? Is added sugar necessary? Can we use artificial mango-candy powder instead of mangoes? Maybe you can hand-wave away each of these, but here's something much worse: what's a mango? You might think that's obvious, a mango is the fruit of mangifera indica or maybe if you're generous fruit of anything in the mangifera genus. But mangoes evolved and there is some weird state where we had almost-a-mango and in the future we might have some new states which are no-longer-a-mango, but more or less we're throwing darts at exactly where we think those are... evolution doesn't care, evolution just wants to keep reproducing.

Meaning changes over time, and how we categorize does too. Once someone was explaining the Web Ontology Language (which got confused somewhere in its acronym ordering and is shortened to OWL (update: it's a Winnie the Pooh update, based on the way the Owl character spells his name... thank you Amy Guy for informing me of the history)). They said that it was great because you could clearly define what is and isn't allowed and terms derived from other terms, and that the simple and classic example is Gender, which is a binary choice of Male or Female. They paused and thought for a moment. "That might not be a good example anymore."

Even if you try to define things by their use or properties rather than as an individual concept, this is messy too. A person from two centuries ago would be confused by the metal cube I call a "stove" today, but you could say it does the same job. Nonetheless, if I asked you to "fetch me a stove", you would probably not direct me to a computer processor or a car engine, even though sometimes people fry an egg on both of these.

Multiple constructed languages (Esperanto most famously) have been made by authors that believed that if everyone spoke the same language, we would have world peace. This is a beautiful idea, that conflict comes purely from misunderstandings. I don't think it's true, especially given how many fights I've seen between people speaking the same language. Nonetheless there's truth in that many fights are about a conflict of ideas.

If anyone was going to achieve this though, it would be the Lojban community, which actually does have a language which is syntactically unambiguous, so you no longer have ambiguity such as "time flies like an arrow". Nonetheless, even this world can't escape the problem that some terms just can't be easily pinned down, and the best example is the bear goo debate.

Here's how it works: both of us can unambiguously construct a sentence referring to a "bear". But when it is that bear no longer a bear? If it is struck in the head and is killed, when in that process has it become a decompositional "bear goo" instead? And the answer is: there is no good answer. Nonetheless many participants want there to be a pre-defined bear, they want us to live in a pre-designed universe where "bear" is a clear predicate that can be checked against, because the universe has a clear definition of "bear" for us.

That doesn't exist, because bears evolved. And more importantly, the concept and existence a bear is emergent, cut across many different domains, from evolution to biology to physics to linguistics.

Sorry, we won't achieve perfect communication, not even in Lojban. But we can get a lot better, and set up a system with fewer stumbling blocks for testing ideas against each other, and that is a worthwhile goal.

Nonetheless, if you and I are camping and I shout, "AAH! A bear! RUN!!", you and I probably don't have to stop to debate bear goo. Rees is right that language is a reverse engineering effort, but we tend to do a pretty good job of gaining rough consensus of what the other side means. Likewise, if I ask you, "Where is your stove?", you probably won't lead me to your computer or your car. And if you hand me a "sugar free vegan mango lassi made with artificial mango flavor" I might doubt its cultural authenticity, but if you then referred to the "mango lassi" you had just handed me a moment ago, I wouldn't have any trouble continuing the conversation. Because we're more or less built to contextually construct language contexts.

Language is a quadratic explosion of Katamaris

Language is composed of syntax partly, but the arrangement of symbolic terms mostly. Or that's another way to say that the non-syntactic elements of language are mostly there as identifiers substituted mentally for identity and all the associations therein.

Back to the Katamari metaphor. What "language is a reverse-engineering effort" really means is that each of us are constructing identities for identifiers mentally, rolling up katamaris for each identifier we encounter. But what ends up in our ball will vary depending on our experiences and what paths we take.

Which really means that if each person is rolling up a separate, personal identity-katamari for each identifier in the system, that means that, barring passing through a singularity type event-horizon past which participants can do direct shared memory mapping, this is an O(n^2) problem!

But actually this is not a problem, and is kind of beautiful. It is amazing, given all that, just how good we are at finding shared meaning. But it also means that we should be aware of what this means topologically, and that each participant in the system will have a different set of experiences and understanding for each identity-assertion made.

Thank you to Morgan Lemmer-Webber, Stephen Webber, Corbin Simpson, Baldur Jóhannsson, Joey Hess, Sam Smith, Lee Spector, and Jonathan Rees for contributing thoughts that lead to this post (if you feel like you don't belong here, do belong here, or are wondering how the heck you got here, feel free to contact me). Which is not to say that everyone, from their respective positions, have agreement here; I know several disagree strongly with me on some points I've made. But everyone did help contribute to reverse-engineering their positions against mine to help come to some level of shared understanding, and the giant pile of katamaris that is this blogpost.


MyDigitalFootprint

Revising the S-Curve in an age of emergence

Exploring how the S-Curve can help us with leadership, strategy and decisions making in an age of emergence: (properties or behaviours which only emerge when the parts interact as part of an inclusive whole) History and context There is a special place in our business hearts and minds for the “S” curve or Sigmoid function, calling it by its proper maths name. The origin of the S curve goes back
Exploring how the S-Curve can help us with leadership, strategy and decisions making in an age of emergence: (properties or behaviours which only emerge when the parts interact as part of an inclusive whole) History and context

There is a special place in our business hearts and minds for the “S” curve or Sigmoid function, calling it by its proper maths name. The origin of the S curve goes back to the study of population growth by Pierre-François Verhulst c.1838. Verhulst was influenced by Thomas Malthus’ “An Essay on the Principle of Population” which showed that growth of a biological population is self-limiting by the finite amount of available resources. The logistic equation is also sometimes called the Verhulst-Pearl equation following its rediscovery in 1920. Alfred J. Lotka derived the equation again in 1925, calling it the law of population growth but he is better known for his predator: prey model.  

In 1957 business strategists Joe Bohlen and George Beal published the Diffusion Process. Taking the adoption curve and adding cumulatively the take up the product to gain a “classic S curve.”  


The market adoption curve became the basis for explaining innovation and growth as a broader market economic concept by the late 1960s. We started to consider the incubation of ideas to create new businesses and how we need a flow of products/ services within big companies.

From this thinking emerged two concepts, as the shareholder primacy model became central to growth.  The first is the concept of “Curve Jumping” to ensure that you continue growth by keeping shareholders happy through the continuous introduction of new products, as the existing ones have matured. Of course, the downside is that if a business cannot jump because of its current cost base or ability to adopt the latest technology to perpetuate the ascension of the curve, new companies will emerge with competitive advantages (product or cost) as they jump to new technologies.  Milton Friedman’s emphasis on shareholder value maximisation at the expense of other considerations was driving companies to keep up with the next curve of fear of being left behind competitively. Some sorts of competition are healthier for markets than others, and it appears that competition and anxiety relating to retaining technology leadership at all costs have been driving capitalism in a particularly destructive direction, rather than encouraging useful and sustainable friendly innovation.  There is an economics essay to be written here, but this piece is about the S curve.


 

Right here, right now.

We live in a time when crisis and systematic “emergent properties” are gaining attention and prominence.  Emergence by definition occurs when an entity or system is observed to have properties that its parts do not pose or display on their own. Properties or behaviours which only emerge when the parts interact in the broader system, as we see our businesses, we understand as complex adaptive systems. 

Whilst shareholder primacy as an economic driver faded in 1990 to be replaced finally in 2019 with Colin Mayer work on the Purpose Model, a modern standard for corporate responsibility which makes an equal commitment to all stakeholders. Shareholder primacy’s simplicity has remained a stalwart of leadership training, teaching and therefore, management thinking.  Its simplicity meant we did not have to deal with contradictions and conflicting requirements that a broader purpose would expose. The Business Round Table Aug 2019 and Blackrock's CEO Larry Fink letters to CEOs/ Shareholders are critical milestones in turning thinking away from pure shareholder returns as the reason for a business to exist. The shift is towards eco-systems and ESG (Environmental sustainability, Social responsibility and better oversight and Governance) as primary drives.  The FCA Stewardship code, Section 172 of the companies act and decision reporting are some of the first legislative instruments on this journey. With now over 50 series A funded startups active in ESG reporting, impact investing has become a meme as the development of a more standardised and comprehensive purpose reporting has strengthened.   

Shareholder primacy’s simplicity meant we did not have to deal with contradictions and conflicting requirements that a broader purpose would expose.

With this new framing, it is time to revisit the S-Curve. 

Framing the S-Curve for an evolutionary journey

If you have not yet discovered Simon Wardley and his mapping thinking, stop here and watch this.   Simon has a brilliant S-Curve with pioneers, settlers and town planners, really worth looking up. His model is about evolution (journey towards commodity) rather than diffusion (take up over time).  To quote Simon “The evolution of a single act from genesis to commodity may involve hundreds if not thousands of diffusion curves for each evolving instance of that act, each with their own chasm.”

The next S-Curve below, I am building on Simon’s axis of Ubiquity (how common something is) and Certainty (the likelihood of the outcome being as determined) which is an evolution S-Curve.  On these axes, we are plotting the systems and company development - time is not present, but as we will see, we have to remove time from the framing.  

Starting in the bottom left is the activity of Innovation where uniquity is low, as it is an idea, and it is not available to everyone. The certainty that any innovation will work is low.

  


The top right corner is the perceived north star.  Ubiquity is high as everyone has it, and there is high certainty that it works.  In this top right corner are commodities and utilities, technology at scale, by example turning on the water tap and drinking water flows.  Linking innovation and commodity is an evolution or journey S-Curve. Under this curve, we can talk about the transformation of the company, the companies practices, data, controls and what models it will most likely utilise.  The chart below highlights the most popular thinking in each stage and is certainly not exclusive.  Agile works well in all phases, AI can be used anywhere, except for choice, and data is not as definite as the buckets would suggest.  Control changes as we evolve from lean/ MVP in the first delivery, to using methodologies such as agile and scrum, then Prince 2 as a grown-up project management tool at scale and then towards quality management with 6 Sigma. 

Note: I have a passionate dislike of the term “best practice” as it only applies when in the linear phase but is literally applied everywhere.  At linear, you have evidence and data to support what “best” looks like.  At any stage before ubiquity and certainty, best practice is simply not possible other than by lucking out.  A desire for best practice ignores that you have to learn and prove what is it before you find it is best. And to all those who have the best - you can still do better, so how can it be best?

If one considers the idea of time and S-Curves you get to curve jumping or continual product development as set out earlier.  The purpose of an evolution or journey S-Curve presented in this way is that when time is not the axis, any significant business will have all these activities present at all times (continual adaptation/ evolution not diffusion). In nature, all different levels of species exist at the same time from a single cell, to complex organisms.  Innovation is not a thing; it is a journey, where you have to be all the camps, on the route, at the same time. 

Innovation is not a thing; it is a journey where you have to be all the camps, on the route, at the same time. 

Evolution S-Curve and governance

HBR argues that most capitalists markets are in a post-shareholder primacy model, meaning the purpose of an organisation is up for debate. Still, we are on the route to a more inclusive purpose and reason for a company to exist.  Law already exists in the UK as Section 172 of the Companies Act in the form of directors duties.  The global pandemic has highlighted a bunch of significant weakness that emerges from our focus on growth and shareholder focus, including, as examples only:

Highly integrated long supply chains are incredibly efficient, but are very brittle and not resilient  - we lost effectiveness.

A business needs to re-balance towards effectiveness.  A food company in a pandemic exists to get food to customers (effectiveness) not to drive efficiency at any cost.

Ecosystem sustainability is more important than any single company's fortunes.

ESG, risk, being better ancestors, costing the earth and climate change are extremely difficult on your own.

Our existing risk models focus on resource allocation, safety and control. This framing means that new risk created in a digital-first world may be outside of the frame and therefore hidden in plain sight. 

Given this framing and context, it is worth overlaying governance on the S-curve of start-up development, which we will now unpack.  

Governance has centrally focussed on corporates and large companies who offer products and services to mass markets. By concentrating governance on companies who have scale, if they are well managed, and is there independence of oversight, we have framed governance as only of interest for companies where there is an interest to wider society on their behaviour. Indeed it becomes a burden rather than a value. 

Companies of scale tend to be found in the linear quadrant, top right, where growth is mainly incremental and linear.  Regulation and markets focus on “BEST practices” which have been derived over a long period. The data used is highly modelled, and the application of AI creates new opportunities and value.  Control is exercised through the utilisation of 6 Sigma for quality (repeatability) and other advanced program management techniques. KPI’s enable the delegation of actions and the monitoring, control thereof.  The business model is that of exercising good or “best” decision making, based on resource allocation and risk.   

Unpacking Corporate Governance is a broad and thorny topic, but foundations such as The Cadbury Report (1992) and the Sarbanes–Oxley Act (2002) have been instrumental in framing mandates.  However, governance, compliance and risk management became one topic in c.2007 and lost the clear separation of function.  Regulation has also formed an effective backstop to control behaviours and market abuse.  

The point is, when a company is at the scale, we have created “best governance practises and guidance”,  along with excellent risk frameworks and stewardship codes for investors.   Many of the tools and methods have stood the test of time and provide confidence to the market.  However, these tools and frameworks are designed for companies at scale.  On the journey from startup to scale, the adoption of such heavyweight practices in early development would be overly burdensome for emergent companies and are not a best or a good fit.  

Remembering that any company of scale has all these phases present at the same time, but there are five possible camps or phases where we need governance; three are in orange and two in yellow.   The yellow blocks represent phases where there is a degree of stability insomuch that there can be growth, but there is not a wholesale change in everything.  The orange block represents phases where everything is changing.   Yellow blocks indicate complicated oversight,  where orange suggests complex.  

To be clear, it is not that companies or markets in a linear phase are not complex; it is that the management at linear has more certainty in terms of practices and forecasting coupled with having to deal with less change. When there is a product of service at linear it delivers significant noisy signals and priorities that often overshadow any data or insights from other phases.  Management at scale requires a focus on understanding the delta between the plan and the executed outcome and making minor adjustments to optimise.  

The management during the yellow stable growth camps/ phases is complicated as patterns and data will not always be that useful.  Data will not be able to point to a definitive decision directly. Governance provides assurance and insights as management continually searches for the correct data to make decisions on, which may not be there.  Management during an orange highly volatile camps/ phases is more complicated as you cannot rely on existing data during a transition between what you had and the new. Simply put if you did, you will only get what you have and not be able to move to the new.  The idea of transition is that the old is left behind. Experienced leadership will show skills in seeking small signals from the noisy existing data and the noise. When considering governance through this dynamic lens, it is apparent that it becomes much more challenging and that we cannot rely on the wisdom and best practices of linear. 

Plans at scale are more comfortable and more predictable; they are designed to enable the measurement of a delta. Plans during innovation are precisely the opposite, not easy and highly unpredictable.  Using the same word “plan” in both cases means we lose definition and distinction.  

A plan at scale is built on years of data and modelled to be highly reliable; it is accurate and has a level of detail that can create KPIs for reporting.  The plan and the model is a fantastic prediction tool.  

A plan at start-up and growth is about direction and intention. Failure to have one would be catastrophic,  but with the first few hours of the words being committed to a shared document, the plan is out of date.  To be useful, it has to lack precision, detail and measurement but will set out stages, actions and outcomes.  It must have a purpose, direction and how to frame complex decisions. 

Similarly, governance at scale is more comfortable and more predictable; governance is about understanding where and how delta might arise and be ready for it. Governance during innovation is precisely the opposite, not easy and highly unpredictable.  Using the same word “Governance” in both cases means we lose definition and distinction.

Using the same word “Governance” at scale and in startup cases means we lose definition and distinction.    

Complexity: Organisational mash-ups

Many businesses are mash-ups of previous transformations, plus current evolution. This observation has two ramifications: One, the structure, processes and skills are neither fully aligned to the original model or various constructions of a new model.  Two, data shows that as you categorise focus and alignment in the more senior positions, and who have been in post longer, most have a compass or alignment coupled with a mash-up of a previous model. Bluntly they stopped on the evolution path creating a dead end.  Senior management who tend to have a closed mindset, rather than an open and continually learning one, tend to fall back on the experience of previous best practices, models and pre-transformational ideals, adding a significant burden to governance for any stage.  The concept that there is a direct coupling between innovation and KPI measurement, which makes it harder for corporates to innovate and evolve is explored in this article.   

All companies have an increasing dependence on ecosystems for their growth and survival.  Ecosystem health is critical for companies at scale for supply chains and customers.  Companies who operate at scale and in the linear phase, therefore, are dependent on companies who are in different stages on a planned route to scale. Thus, not only is a scale company dealing with its internal governance and innovation requirements as already noted, but the directors have to understand data from an ecosystem, who is also trying to understand what their data is telling them about their evolution path.  

Directors have to understand data from an ecosystem, who is also trying to understand what their data is telling them about their evolution path.  

Governance is not about best practices and processes at any stage; it is about a mindset of an entire organisation and now ecosystem.  When you reflect on it, Directors with governance responsibilities have to cope with data for decisions from chaotic and linear requirements at the same time — equally relying on individuals and teams who have different perceptions both inside and outside of the organisation. Never has data-sharing been more important as a concept, both as a tool or weapon (inaccurate data) in competitive markets.   How can a director know that the data they get from their ecosystem can support their decision making and complex judgement?

Take Away

The S-curve has helped us on several journeys thus far. It supported our understanding of adoption and growth; it can now be critical in helping us understand the development and evolution of governance towards a sustainable future.  An evolutionary S-curve is more applicable than ever as we enter a new phase of emergence.  Our actions and behaviours emerge when we grasp that all parts of our ecosystem interact as a more comprehensive whole. 

A governance S-curve can help us unpack new risks in this dependent ecosystem so that we can make better judgements that lead to better outcomes. What is evident is that we need far more than proof, lineage and provenance of data from a wide ecosystem if we are going to create better judgement environments, we need a new platform. Such a new platform is my focus and why I am working on Digital20.  


Friday, 04. December 2020

Aaron Parecki

IndieAuth Spec Updates 2020

This year, the IndieWeb community has been making progress on iterating and evolving the IndieAuth protocol. IndieAuth is an extension of OAuth 2.0 that enables it to work with personal websites and in a decentralized environment.

This year, the IndieWeb community has been making progress on iterating and evolving the IndieAuth protocol. IndieAuth is an extension of OAuth 2.0 that enables it to work with personal websites and in a decentralized environment.

There are already a good number of IndieAuth providers and apps, including a WordPress plugin, a Drupal module, and built-in support in Micro.blog, Known and Dobrado. Ideally, we'd have even more first-class support for IndieAuth in a variety of different blogging platforms and personal websites, and that's been the goal motivating the spec updates this year. We've been focusing on simplifying the protocol and bringing it more in line with OAuth 2.0 so that it's both easier to understand and also easier to adapt existing OAuth clients and servers to add IndieAuth support.

Most of the changes this year have removed IndieAuth-specific bits to reuse things from OAuth when possible, and cleaning up the text of the spec. These changes are also intended to be backwards compatible as much as possible, so that existing clients and servers can upgrade independently.

This post describes the high level changes to the protocol, and is meant to help implementers get an idea of what may need to be updated with existing implementations.

If you would like an introduction to IndieAuth and OAuth, here are a few resources:

IndieAuth: OAuth for the Open Web OAuth 2.0 Simplified OAuth 2.0 Simplified, the book indieauth.net

The rest of this post details the specifics of the changes and what they mean to client and server developers. If you've written an IndieAuth client or server, you'll definitely want to read this post to know what you'll need to change for the latest updates.

Response Type Indicating the User who is Logging In Adding PKCE Support Grant Type Parameters Providing "me" in the Token Request Removing Same-Domain Requirement Returning Profile Information Editorial Changes Dropped Features and Text Response Type

The first thing an IndieAuth client does is discover the user's authorization endpoint and redirect the user to their server to authorize the client. There are two possible ways a client might be wanting to use IndieAuth, either to confirm the website of the user who just logged in, or to get an access token to be able to create posts on their website.

Previously, this distinction was made at this stage of the request by varying the response_type query string parameter. Instead, the response_type parameter is now always response_type=code which brings it in line with the OAuth 2.0 specification. This makes sense because the response of this request is always an authorization code, it's only after the authorization code is used that the difference between these two uses will be apparent.

Changes for clients: Always send response_type=code in the initial authorization request.

Changes for servers: Only accept response_type=code requests, and for backwards-compatible support, treat response_type=id requests as response_type=code requests.

Indicating the User who is Logging In

In earlier versions of the specification, the authorization request was required to have the parameter me, the value of which was whatever URL the user entered into the client to start the flow. It turns out that this parameter isn't strictly necessary for the flow to succeed, however it still can help improve the user experience in some cases. As such, it has now been changed to an optional parameter.

This parameter is a way for the client to tell the IndieAuth server which user it expects will log in. For single-user sites this value is completely unnecessary, since there is only ever one me URL that will be returned in the end. It turns out that most single-user implementations were already ignoring this parameter anyway since it served no purpose.

For multi-user websites like a multi-author WordPress blog, this parameter also served little purpose. If a user was already logged in to their WordPress site, then tried to log in to an IndieAuth client, the server could just ignore this parameter anyway and return the logged-in user's me URL at the end of the flow.

For multi-user authorization endpoints like the (to-be deprecated) indieauth.com, this parameter served as a hint of who was trying to log in, so that the authorization server could provide a list of authentication options to the user. This is the only case in which this parameter really provides a user experience benefit, since without the parameter at this stage, the user would need to type in their website again, or be shown a list of authentication provider options such as "log in with Twitter".

There's yet another case where the user may enter just the domain of their website, even though their final me URL may be something more specific. For example, a user can enter micro.blog in an IndieAuth sign-in prompt, and eventually be logged in to that app as https://micro.blog/username. There is no requirement that the thing they type in to clients has to be an exact match of their actual profile URL, which allows a much nicer user experience so that users can type only the domain of their service provider which may provide profiles for multiple users. And in this case, the client providing the me URL to the server also doesn't serve any purpose.

The change to the spec makes the me parameter in the authorization request optional. In this sense, it's more of a hint from the client about who is trying to log in. Obviously the server can't trust that value in the request at this point, since the user hasn't logged in yet, so it really is more of a hint than anything else.

Changes for clients: Continue to include the me parameter in the request if you can, but if you are using an OAuth client that doesn't let you customize this request, it's okay to leave it out now.

Changes for servers: Most servers were already ignoring this parameter anyway, so if you fell into that category then no change is needed. If you were expecting this parameter to exist, change it to optional, because you probably don't actually need it. If it's present in a request, you can use it to influence the options you show for someone to authenticate if they are not yet logged in, or you could show an error message if the client provides a me URL that doesn't match the currently logged-in user.

Adding PKCE Support

Probably the biggest change to the spec is the addition of the OAuth 2.0 PKCE (Proof Key for Code Exchange) mechanism. This is an extension to OAuth 2.0 that solves a number of different vulnerabilities. It was originally designed to allow mobile apps to securely complete an OAuth flow without a client secret, but has since proven to be useful for JavaScript apps and even solves a particular attack even if the client does have a client secret.

Since IndieAuth clients are all considered "Public Clients" in OAuth terms, there are no preregistered client secrets at all, and PKCE becomes a very useful mechanism to secure the flow.

I won't go into the details of the particular attacks PKCE solves in this post, since I've talked about them a lot in other talks and videos. If you'd like to learn more about this, check out this sketch notes video where I talk about PKCE and my coworker draws sketchnotes on his iPad.

Suffice it to say, PKCE is a very useful mechanism, isn't terribly complicated to implement, and can be added independently by clients and servers since it's designed to be backwards compatible.

The change to the spec is that PKCE has been rolled into the core authorization flow. Incidentally, the OAuth spec itself is making the same change by rolling PKCE in to the OAuth 2.1 update.

Changes for clients: Always include the PKCE parameters code_challenge and code_challenge_method in the authorization request.

Changes for servers: If a code_challenge is provided in an authorization request, don't allow the authorization code to be used unless the corresponding code_verifier is present in the request using the authorization code. For backwards compatibility, if no code_challenge is provided in the request, make sure the request to use the authorization code does not contain a code_verifier.

Using an Authorization Code

Whew, okay, you've made it this far and you've sent the user off to their authorization endpoint to log in. Eventually the IndieAuth server will redirect the user back to your application. Now you're ready to use that authorization code to either get an access token or confirm their me URL.

There are two changes to this step, redeeming the authorization code.

Grant Type Parameters

The first change, while minor, brings IndieAuth in line with OAuth 2.0 since apparently this hadn't been actually specified before. This request must now contain the POST body parameter grant_type=authorization_code.

Changes for clients: Always send the parameter grant_type=authorization_code when redeeming an authorization code. Generic OAuth 2.0 clients will already be doing this.

Changes for servers: For backwards compatibility, treat the omission of this parameter the same as providing it with grant_type=authorization_code. For example if you also accept requests with grant_type=refresh_token, the absence of this parameter means the client is doing an authorization code grant.

Providing "me" in the Token Request

The request when using an authorization code, either to the token endpoint or authorization endpoint, previously required that the client send the me parameter as well. The change to the spec drops this parameter from this request, making it the same as an OAuth 2.0 request.

This has only some minor implications in very specific scenarios. We analyzed all the known IndieAuth implementations and found that the vast majority of them were already ignoring this parameter anyway. For single-user endpoints, the additional parameter provides no value, since the endpoint would be self-contained anyway, and already know how to validate authorization codes. Even multi-user endpoints like the WordPress plugin would know how to validate authorization codes because the authorization and token endpoints are part of the same software.

The only implementations leaving this parameter out would break are separate implementations of authorization endpoints and token endpoints, where the user has no prior relationship with either. The biggest offender of this is actually my own implementation which I am eventually going to retire, indieauth.com and tokens.indieauth.com. I initially wrote indieauth.com as just the authorization endpoint part, and later added tokens.indieauth.com as a completely separate implementation, it shares nothing in common with indieauth.com and is actually entirely stateless. Over the years, it turns out this pattern hasn't actually been particularly useful, since a website is either going to build both endpoints or delegate both to an external service. So in practice, the only people using tokens.indieauth.com were using it with the indieauth.com authorization endpoint.

Removing this parameter has no effect on most of the implementations. I did have to update my own implementation of tokens.indieauth.com to default to verifying authorization codes at indieauth.com if there was no me parameter, which so far has been sccessful.

Changes for clients: No need to send the me parameter when exchanging an authorization code. This makes the request the same as a generic OAuth 2.0 request.

Changes for servers: For servers that have an authorization endpoint and token endpoint as part of the same software, make sure your token endpoint knows how to look up authorization codes. Most of the time this is likely what you're already doing anyway, and you were probably ignoring the me parameter already. If you do want to provide a standalone token endpoint, you'll need to create your own encoding scheme to bake in the authorization endpoint or me value into the authorization code itself. But for the vast majority of people this will require no change.

Removing Same-Domain Requirement

One of the challenges of a decentralized protocol like this is knowing who to trust to make assertions about who. Just because someone's authorization server claims that a user identified as "https://aaronpk.com/" logged in doesn't mean I actually did log in. Only my authorization server should be trusted to assert that I logged in.

In the previous version of the spec, the way this was enforced was that clients had to check that the final me URL returned had a matching domain as what the user initially entered, after following redirects. That means if I entered aaronpk.com into the client, and that redirected to https://aaronparecki.com/, the client would then expect the final profile URL returned at the end to also be on aaronparecki.com. This works, but it has a few challenges and limitations.

The biggest challenge for client developers was keeping track of the chain of redirects. There were actually separate rules for temporary vs permanent redirects, and the client would have to be aware of each step in the redirect chain if there was more than one. Then at the end, the client would have to parse the final profile URL to find the host component, then check if that matches, and it turns out that there are often some pretty low-level bugs with parsing URLs in a variety of languages that can lead to unexpected security flaws.

On top of the technical challenges for client developers, there was another problem in the specific case where a user may control only a subfolder of a domain. For example in a shared hosting environment where users can upload arbitrary files to their user directory, https://example.com/~user, the same-domain restriction would still let /~user1 claim to be /~user2 on that domain. We didn't want to go down the route of adding more URL parsing rules like checking for substring matches, as that would likely have led to even more of a burden on client developers and more risk of security holes.

So instead, this entire restriction has been replaced with a new way of verifying that the final profile URL is legitimate. The new rule should drastically simplify the client code, at the slight cost of a possible additional HTTP request.

The new rule is that if the final profile URL returned by the authorization endpoint is not an exact match of the initially entered URL, the client has to go discover the authorization endpoint at the new URL and verify that it matches the authorization endpoint it used for the flow. This is described in a new section of the spec, Authorization Server Confirmation.

This change means clients no longer need to keep track of the full redirect chain (although they still can if they would like more opportunities to possibly skip that last HTTP request), and also ensures users on shared domains can't impersonate other users on that domain.

Changes for clients: Remove any code around parsing the initial and final URLs, and add a new step after receiving the user's final profile URL: If the final profile URL doesn't match exactly what was used to start the flow, then go fetch that URL and discover the authorization endpoint and confirm that the discovered authorization endpoint matches the one used at the beginning. Please read Authorization Server Confirmation for the full details.

Changes for servers: No change.

Returning Profile Information

If the application would like to know more about the user than just their confirmed profile URL, such as their name or photo, previously there was no easy or reliable way to find this information. It's possible the user's profile URL may have an h-card with their info, but that would only include public info and would require bringing in a Microformats parser and making another HTTP request to find this information.

In the latest version of the spec, we've added a new section returned in the response when redeeming an authorization code for the authorization server to return this profile data directly. To request this information, there are now two scopes defined in the spec, profile and email. When the client requests the profile scope, this indicates the client would like the server to return the user's name, photo and url. The email scope requests the user's email address.

The response when redeeming an authorization code that was issued with these scopes will now contain an additional property, profile, alongside the me URL and access token.

{ "access_token": "XXXXXX", "token_type": "Bearer", "scope": "profile email create", "me": "https://user.example.net/", "profile": { "name": "Example User", "url": "https://user.example.net/", "photo": "https://user.example.net/photo.jpg", "email": "user@example.net" } }

This comes with some caveats. As is always the case with OAuth, just because a client requests certain scopes does not guarantee the request will be granted. The user or the authorization server may decide to not honor the request and leave this information out. For example a user may choose to not share their email even if the app requests it.

Additionally, the information in this profile section is not guaranteed to be "real" or "verified" in any way. It is merely information that the user intends to share with the app. This means everything from the user sharing different email addresses with different apps, or the URL in the profile being a completely different website. For example a multi-author WordPress blog which provides me URLs on the WordPress site's domain, example.com, may return the author's own personal website in the url property of the profile information. The client is not allowed to treat this information as authoritative or many any policy decisions based on the profile information, it's for informational purposes only. Another common vulnerability in many existing OAuth clients is that they assume the provider has confirmed the email address returned and will use that to deduplicate accounts. This has the problem of if a user can edit their email address and have it returned in an OAuth response without the server confirming it, the client may end up being tricked into thinking a different user logged in. Only the me URL is the one that can be trusted as the stable identifier of the user, and everything in the profile section should be treated as if it were hand-entered into the client.

Changes for clients: If you would like to find the user's profile information, include the profile or email scope in your authorization request. If you don't need this, then no changes are necessary.

Changes for servers: Authorization servers should be able to recognize the profile and email scopes in a request, and ask the user for permission to share their profile information with clients, then return that along with the final me URL and access token. It's also completely acceptable to not support this feature at all, as clients shouldn't be relying on the presence of this information in the response anyway.

Editorial Changes

There was a good amount of work done to clean up the text of the spec without changing any of the actual requirements. These are known as editorial changes.

The term "domain" has been replaced with the more accurate term "host" in most places. This matches the URL spec more closely, and reduces the confusion around registerable domain like example.com or example.co.uk and subdomains. In all cases, there has been no need to use the public suffix list because we have always meant full hostname matches.

Language around the term "profile URL" was cleaned up to make sure only the final URL returned by the authorization server is referred to as the "profile URL". The user may enter lots of different things into the client that might not be their profile URL, anything from just a hostname (aaronpk.com) to a URL that redirects to their profile URL. This cleans up the language to better clarify what we mean by "profile URL".

With the change to use response_type=code for both versions of the flow, it meant the authorization and authentication sections were almost entirely duplicate content. These have been consolidated into a single section, Authorization, and the only difference now is the response when the authorization code is redeemed.

Dropped Features and Text

Any time you can cut text from a spec and have it mean the same thing is a good thing. Thankfully we were able to cut a decent amount of text thanks to consolidating the two sections mentioned above. We also dropped an obscure feature that was extremely under-utilized. For the case where a token endpoint and authorization endpoint were not part of the same software, there was a section describing how those two could communicate so that the token endpoint could validate authorization codes issued by an arbitrary authorization endpoint. This serves no purpose if a single piece of software provided both endpoints since it would be far more efficient to have the token endpoint look up the authorization code in the database or however you're storing them, so virtually nobody had even bothered to implement this.

The only known implementations of this feature were my own tokens.indieauth.com, and Martijn's mintoken project. We both agreed that if we did want to pursue this feature in the future, we could write it up as an extension. Personally I plan on shutting down indieauth.com and tokens.indieauth.com in the near-ish future anyway, and the replacement that I build will contain both endpoints anyway, so I don't really plan on revisiting this topic anyway.

Conclusion / Future Work

Well if you've made it this far, congrats! I hope this post was helpful. This was definitely a good amount of changes, although hopefully all for good reasons and should simplify the process of developing IndieAuth clients and servers in the future.

We didn't get to every open IndieAuth issue in this round of updates, there are still a few interesting ones open that I would like to see addressed. The next largest change that will affect implementations would be to continue to bring this in line with OAuth 2.0 and always redeem the authorization code at the token endpoint even if no access token is expected to be returned. That would also have the added benefit of simplifying the authorization endpoint implementation to only need to worry about providing the authorization UI, leaving all the JSON responses to the token endpoint. This still requires some discussion and a plan for upgrading to this new model, so feel free to chime in on the discussions!

I would like to give a huge thank-you to everyone who has participated in the discussions this year, both on GitHub and in our virtual meetings! All the feedback from everyone who is interested in the spec has been extremely valuable!

We'll likely schedule some more sessions to continue development on the spec, so keep an eye on events.indieweb.org for upcoming events tagged #indieauth!

If you have any questions, feel free to stop by the #indieweb-dev chat (or join from IRC or Slack) and say hi!

Thursday, 03. December 2020

SSI Ambassador

The mental models of identity enabled by SSI

This article takes the mental models of identity and explores how they can be achieved with a self-sovereign identity (SSI) solution. To pin down the meaning and definition of identity is a challenging task due to its uniquely human nature. It can have totally different meanings for different people. However, there are reoccurring themes when speaking about the term. The following five mental mod

This article takes the mental models of identity and explores how they can be achieved with a self-sovereign identity (SSI) solution.

To pin down the meaning and definition of identity is a challenging task due to its uniquely human nature. It can have totally different meanings for different people. However, there are reoccurring themes when speaking about the term. The following five mental models describe what people refer to, when speaking about identity and provide a useful structure of how these models can be executed in a digital environment leveraging SSI infrastructure and components. While the concept of SSI can be applied for individuals, legal entities and things alike, the following paragraph solely focuses on individuals and explains how these models can serve as a guideline for SSI implementations. The five mental models were published by experts of the RWOT community and are quoted in the following paragraphs.

Mental models of identity. Image source: Lissi Space-time

“The space-time mental model sees identity as resolving the question of the physical continuity of an entity through space and time. (…) It answers the question: Does the physical body under evaluation have a continuous link through space and time to a known entity?”

An identity is established in the past, it acts in the present and continues to be useful in the future. To secure the sum of recorded interactions and relationships in digital form one requires a backup when using a wallet, which stores the identity data and their associated cryptographic keys locally on the device of the user. This backup enables the user to restore the received credentials as well as established relationships. When losing access to the wallet, the backup enables the user to reestablish the aspects described in the space-time mental model. A backup generally consists of the identity data itself and a key, which is used to en- and decrypt the backup data.

Presentation

“The presentation mental model sees identity as how we present ourselves to society. This is the mental model behind Vendor Relationship Management, user-centric identity, and self-sovereign identity. (…) It answers the question: Is this how the subject chooses to be known?”

Individuals can choose, which information about them should be known by third parties or the public. The granularity of this information varies dependent on the social context. While one might only want to provide the required minimum of information to a government authority, one might have the desire to share very personal details with a certain social circle such as family or friends. Hence, the user requires different social profiles and circles, which help to present the right information to the target audience. Since one part of a SSI ecosystem is the creation of trusted peer to peer relationships, these contacts can be sorted by the user and allocated to a social circle according to the preferences of the individual.

However, when it comes to the sharing of information it gets tricky. There are currently no SSI implementations with enable a user-experience similar to current social media platforms. Hence, the presentation of information is currently limited to one contact at a time.

Attribute

“The attribute mental model sees identity as the set of attributes related to an entity as recorded in a specific system. Enshrined in ISO/IEC 24760–1, an international standard for identity management, this mental model is the primary focus for many engineers. (…) It answers the question: Who is this data about?”

From a birth certificate to a university degree or a language certification, we collect a variety of credentials, which attest certain information about us. The sum of all these credentials can also be seen as one mental model of identity. These credentials are issued, stored and managed by the individual and are standardized within the specification of the verifiable credentials data model 1.0 by the W3C. It is the only mental model with a formal specification.

SSI implementations use cryptography to provide the necessary proofs that presented information is about the individual in question. There are different options of implementations to ensure that a certain identifier relates to the specific person, however most implementations use decentralised identifiers (DIDs) to identify the identity subject.

Relationship

“The relationship mental model sees identity emerging through interactions and relationships with others. Our identity is not about what we are in isolation from others, but is rather defined by the relationships we have. This is the fundamental model in the South African idea of ‘Ubuntu’, meaning ‘I am because we are.’ (…) It answers the question: How is this person related?”

The relationship to other individuals or entities can help to determine the status of a person within society. We can observe different domains of relationships, which depend on the social context like a professional, official, legal, personal, public, business or employment context to name a few. For example a representative of a government like a diplomat has special rights and obligations due to this relationship. Depended on the context, e.g. an interview of said diplomat, it can touch multiple domains by being an official interview, with legal consequences, which is presented to the public and can have a direct effect on the employment relation for the diplomat. Generally, individuals initiate and maintain hundreds or even thousands of relationships to different entities. An SSI solution enables an individual to initiate this relationship by accepting or requesting a connection. Once established this connection serves as communication channel to facilitate the exchange of (verified) information between the two parties. Since both parties are able to validate the identity of the other party it enables the necessary trust in a digital environment. However, a the establishment of a connection isn’t necessary and credentials can also be issued or requested without one. There are special protocols, which standardise the credential exchange and communication between two entities like the DiDcomm protocol.

Capability

“The capability mental model pragmatically defines identity in terms of an individual’s capability to perform some task, including their physical ability now, in the past, or in the future. It is the inevitable approach for anyone in an emergency. (…) It answers the question: What can the subject actually do?”

The primary reason why an identity is required in the online world in the first place are the capabilities that come with it. Without an identity one is still able to browse the web and gather information, however when it comes to online shopping, banking, government applications, employee portals, access control and many other aspects, an identity is necessary to execute those actions. Not all actions require a verified identity. In most cases a self-attested identity is sufficient for the verifier. However, there are multiple cases for which the verifier either has a legitimate interest for only allowing access to verified parties or is obligated by law to verify the identity of an individual. An example for the first case can be access to information for a specific audience like a university, which wants to grant students access to internal documents. The students would not be required to verify their identity every time they want to access the repository, but instead only need to prove that they are a student of said university, without disclosing further personal details. The second case includes telecommunication providers, or financial institutions, which need to comply with know your costumer (KYC) regulations.

Mindmap of the mental models enabled by SSI Mindmap: Mental models of identity enabled by SSI. Full size image here. Source: SSI Ambassador / Adrian Doerk

To conclude it can be said, that all mental models of SSI can be enabled to a certain degree, however when it comes to the space-time (backup) mental model or the presentation (social network) model, we also see that the integration of the concept is quite nascent and requires more development to be comparable with current centralised alternatives.

Disclaimer: This article does not represent the official view of any entity, which is mentioned in this article or which is affiliated with the author. It solely represents the opinion of the author.

SSI Ambassador
Adrian Doerk
Own your keys


Phil Windley's Technometria

Relationships in the Self-Sovereign Internet of Things

Summary: DIDComm-capable agents provide a flexible infrastructure for numerous internet of things use cases. This post looks at Alice and her digital relationship with her F-150 truck. She and the truck have relationships and interactions with the people and institutions she engages as she co-owns, lends and sells it. These and other complicated workflows are all supported by a standards-bas

Summary: DIDComm-capable agents provide a flexible infrastructure for numerous internet of things use cases. This post looks at Alice and her digital relationship with her F-150 truck. She and the truck have relationships and interactions with the people and institutions she engages as she co-owns, lends and sells it. These and other complicated workflows are all supported by a standards-based, open-source, protocol-supporting system for secure, privacy-preserving messaging.

In The Self-Sovereign Internet of Things, I introduced the role that Self-Sovereign Identity (SSI) can play in the internet of things (IoT). The self-sovereign internet of things (SSIoT) relies on the DID-based relationships that SSI provides, and their support for standardized protocols running over DIDComm, to create an internet of things that is much richer, secure, and privacy respecting than the CompuServe of Things we're being offered today. In this post, I extend the use cases I offered in the previous post and discuss the role the heterarchical relationships found in the SSIoT play.

For this post, we're going to focus on Alice's relationship with her F-150 truck and its relationships with other entities. Why a vehicle? Because in 2013 and 2014 I built a commercial connected car product called Fuse that used the relationship-centric model I'm discussing here1. In addition, vehicles exist in a rich, complicated ecosystem that offers many opportunities for interesting relationships. Figure 1 shows some of these.

Figure 1: Vehicle relationships (click to enlarge)

The most important relationship that a car has is with its owner. But there's more than one owner over the car's lifetime. At the beginning of its life, the car's owner is the manufacturer. Later the car is owned by the dealership, and then by a person or finance company. And, of course, cars are frequently resold. Over the course of its lifetime a car will have many owners. Consequently, the car's agent must be smart enough to handle these changes in ownership and the resulting changes in authorizations.

In addition to the owner, the car has relationships with other people: drivers, passengers, and pedestrians. The nature of relationships change over time. For example, the car probably needs to maintain a relationship with the manufacturer and dealer even after they are no longer owners. With these changes to the relationship come changes in rights and responsibilities.

In addition to relationships with owners, cars also have relationships with other players in the vehicle ecosystem including: mechanics, gas stations, insurance companies, finance companies, and government agencies. Vehicles exchange data and money with these players over time. And the car might have relationships with other vehicles, traffic signals, the roadway, and even potholes.

The following sections discuss three scenarios involvoing Alice, the truck, and other people, institutions, and things.

Multiple Owners

One of the relationship types that the CompuServe of Things fails to handle well is multiple owners. Some companies try and others just ignore it. The problem is that when the service provider intermediates the connection to the thing, they have to account for multiple owners and allow those relationships to change over time. For a high-value product, the engineering effort is justified, but for many others, it simple doesn't happen.

Figure 2: Multiple Owners (click to enlarge)

Figure 2 shows the relationships of two owners, Alice and Bob, with the truck. The diagram is simple and hides some of the complexity of the truck dealing with multiple owners. But as I discuss in Fuse with Two Owners some of this is simply ensuring that developers don't assume a single owner when they develop services. The infrastructure for supporting it is built into DIDComm, including standardized support for sub protocols like Introduction.

Lending the Truck

People lend things to friends and neighbors all the time. And people rent things out. Platforms like AirBnB and Outdoorsy are built to support this for high value rentals. But what if we could do it for anything at any time without an intermediating platform? Figure 3 shows the relationships between Alice and her friend Carol who wants to borrow the truck.

Figure 3: Borrowing the Truck (click to enlarge)

Like the multiple owner scenario, Alice would first have a connection with Carol and introduce her to the truck using the Introduction sub protocol. The introduction would give the truck permission to connect to Carol and also tell the truck's agent what protocols to expose to Carol's agent. Alice would also set the relationship's longevity. The specific permissions that the "borrower" relationship enables depend, of course, on the nature of the thing.

The data that the truck stores for different activities is dependent on these relationships. For example, the owner is entitled to know everything, including trips. But someone who borrows the car should be able to see their trips, but not those of other drivers. Relationships dictate the interactions. Of course, a truck is a very complicated thing in a complicated ecosystem. Simpler things, like a shovel might simply be keeping track of who has the thing and where it is. But, as we saw in The Self-Sovereign Internet of Things, there is value in having the thing itself keep track of its interactions, location, and status.

Selling the Truck

Selling the vehicle is more complicated than the previous scenarios. In 2012, we prototyped this scenario for Swift's Innotribe innovations group and presented it at Sibos. Heather Vescent of Purple Tornado created a video that visualizes how a sale of a motorcycle might happen in a heterarchical DIDComm environment2. You can see a screencast of the prototype in operation here. One important goal of the prototype was to support Doc Searls's vision of the Intention Economy. In what follows, I've left out some of the details of what we built. You can find the complete write-up in Buying a Motorcycle: A VRM Scenario using Personal Clouds.

Figure 4: Selling the Truck (click to enlarge)

In Figure 4, Alice is selling the truck to Doug. I'm ignoring how Alice and Doug got connected3 and am just focusing on the sale itself. To complete the transaction, Alice and Doug create a relationship. They both have relationships with their respective credit unions where Doug initiates and Alice confirms the transaction. At the same time, Alice has introduced the truck to Doug as the new owner.

Alice, Doug, and the truck are all connected to the DMV and use these relationships to transfer the title. Doug can use his agent to register the truck and get plates. Doug also has a relationship with his insurance company. He introduces the truck to the insurance company so it can serve as the service intermediary for the policy issuance.

Alice is no longer the owner, but the truck knows things about her that Doug shouldn't have access to and she wants to maintain. We can create a digital twin of the truck that is no long attached to the physical device, but has a copy of all the trip and maintenance information that Alice had co-created with the truck over the years she owned it. This digital twin has all the same functionality for accessing this data that the truck did. At the same time, Alice and Doug can negotiate what data also stays on the truck. Doug likely doesn't care about her trips and fuel purchases, but might want the maintenance data.

Implementation

A few notes on implementation:

The relationships posited in these use cases are all DIDComm-capable relationships. The workflows in these scenarios use DIDComm messaging to communicate. I pointed out several places where the Introduction DIDComm protocol might be used. But there might be other DIDComm protocols defined. For example, we could imagine workflow-specific messages for the scenario where Carol borrows the truck. The scenario where Doug buys the truck is rife with possibilities for protocols on DIDComm that would standardize many of the interactions. Standardizing these workflows through protocol (e.g., a common protocol for vehicle registration) reduces the effort for participants in the ecosystem. Some features, like attenuated permissions on channel are a mix of capabilities. DIDComm supports a Discovery protocol that allows Alice, say, to determine if Doug is open to engaging in a sale transaction. Other permissioning would be done by the agent outside the messaging system. The agents I'm envisioning here are smart, rule-executing agents like those available in picos. Picos provide a powerful model for how a decentralized, heterarchical, interoperable internet of things can be built. Picos provide an DIDComm agent programming platform that is easily extensible. Picos live on an open-source pico engine that can run on anything that supports Node JS. They have been used to build and deploy several production systems, including the Fuse connected-car system discussed above. Conclusion

DIDComm-capable agents can be used to create a sophisticated relationship network that includes people, institutions, things and even soft artifacts like interaction logs. The relationships in that network are rich and varied—just like relationships in the real world. Things, whether they are capable of running their own agents or employ a soft agent as a digital twin, are much more useful when they exist persistently, control their own agent and digital wallet, and can act independently. Things now react and respond to messages from others in the relationship network as they autonomously follow their specific rules.

Everything I've discussed here and in the previous post are doable now. By removing the intermediating administrative systems that make up the CompuServe of Things and moving to a decentralized, peer-to-peer architecture we can unlock the tremendous potential of the Self-Sovereign Internet of Things.

Notes Before Fuse, we'd built a more generalized IoT system based on a relationship network called SquareTag. SquareTag was a social product platform (using the vernacular of the day) that promised to help companies have a relationship with their customers through the product, rather than merely having information about them. My company, Kynetx, and others, including Drummond Reed, were working to introduce something we called "personal clouds" that were arrayed in a relationship network. We built this on a actor-like programming model called "picos". The pico engine and programming environment are still available and have been updated to provide DID-based relationships and support for DIDComm. In 2012, DIDComm didn't exist of course. We were envisioning something that Innotribe called the Digital Asset Grid (DAG) and speaking about "personal clouds" but the envisioned operation of the DAG was very much like what exists now in the DIDComm-enabled peer-to-peer network enabled by DIDs. In the intentcasting prototype, Alice and Doug would have found each other through a system that matches Alice's intent to buy with Doug's intent to sell. But a simpler scenario would have Alice tell the truck to list itself on Craig's List so Alice and Doug can meet up there.

Photo Credit: F-150 from ArtisticOperations (Pixabay License)

Tags: ssiot iot vrm me2b ssi identity decentralized+identifiers relationships


MyDigitalFootprint

Humans want principles, society demands rules and businesses want to manage risk, can we reconcile the differences?

The linkage between principles and rules is not clear because we have created so many words and variances in language that there is significant confusion. We are often confused about what we mean as we are very inconsistent in how we apply words and language, often to provide a benefit to ourselves or justify our belief. To unpack the relationships we need to look at definitions, but we have to a

The linkage between principles and rules is not clear because we have created so many words and variances in language that there is significant confusion. We are often confused about what we mean as we are very inconsistent in how we apply words and language, often to provide a benefit to ourselves or justify our belief. To unpack the relationships we need to look at definitions, but we have to accept that even definitions are inconsistent. Our conformational bias is going to fight us, as we want to believe what we already know, rather than expand our thinking.

(building on orignal article with Kaliyia) Are we imagining principles or values?  

Worth noting our principles are defined by our values. Much like ethics (group beliefs) and morals (personal beliefs) and how in a complex adaptive system my morals affect the group’s ethics and a group’s ethics changes my morals. Situational awareness and experience play a significant part in what you believe right now, and what the group or society believes. 

Values can be adaptable by context whereas principles are fixed for a period, withstanding the test of time.  When setting up a framework where we are setting our principles implies that we are saying that we don’t want them to change every day, week, month, year, that they are good and stable for a generation but we can adapt/ revise/ adjust principles based on learning.  Fundamentally principles are based on values which do change, so there are ebbs and flows of conflict between them, this means we frame principles and often refuse to see that they are not future proof forever.  Indeed the further a principle is away from the time it was created, the less it will have in common with values. 

Are we confusing principles and rules?  

Considering characteristics, conceptually principles are abstract and universal whereas a rule is specific and particular. Principles cope with exceptions, rules need another rule.  Principles provide the power of thought and decision making, rules prevent thought and discretion.  Principles need knowledge and experience to deliver outcomes, rules don’t.  Principles cope with risk, conflict and abstraction; conflict is not possible for a rule, it is this rule or a rule is needed. 


The word “rule” needs some more unpacking as it can take on many meanings.   The history and origin of the word “Rule” is here.  The choice of the word rule is designed to be ambitious, allowing the reader to apply your own context, thereby creating more relevance to your own circumstances. 

For me, you or someone;

Rules are written or unwritten or both

Rules are mine, created by me that you need to follow. They are yours, crafted by you that you need me to obey. They are shared and we believe that they create a better society

Rules can be the law, just a guide, the standard you need to meet or the rituals that creates success.  But which law, they one we should not break or the one where we follow the spirit?  As a guide to guide me from here to where.  As a standard is that absolute or is a range good enough.  My rituals, did I learn them, did you teach me or somehow are they just there?

Rules equally give you more freedom (safety, less murder) and remove your freedom (choice). Rules give me more agency and at the same time remove it.

Rules define my boundaries but are the ones I have created for myself and I have continually refined them as I learn, or are my rules ones that come from history; because we have always done it this way.  

Rules are they creating my view on values or are the rules I have someone else’s values?

Rules are only there to be broken

Rules allow me to create something as I have done something, have experience and have learnt. Rules allow me to repeat and not make the same mistake or improve and adapt.  Rules save me time and energy - I love my heuristics

Rules allow me to manage, prevent and control risk

But whose rules are they?

Back to the relationship between rules and principles.  In companies and for a social policy we set rules and principles into matrices as below.  Asking is it better to break rules or comply, is better to uphold principle or challenge them.  This helps us to define where social norms stop and laws are needed.   

A review round the four quadrants highlights that there is no favourable sector and indeed as a society who wants to get improve, we continually travel through all of them.  Companies and executives often feel that upholding principles and obeyed rules (top right) creates the best culture, but also ask the organisation to be adaptive, agile and innovative. 

Given that principles are based on values, the leadership team will be instrumental into how upheld the principles are. Whereas the companies level of documentation for processes, procedures and rules will define what is to be obeyed, the culture of the top team will determine if they are to be obeyed or not. 

The matrix below thinks about the combinations of values and principles. Where values are either mine as an individual or we as a collective society.  



The fundamental issue with the two representations (rules or values and principles)  is that they cannot highlight the dynamic nature of the relationship between them.  By example, our collective values help normalise an individuals bias and that collective values informs and refine principles.  Indeed as principles become extreme and too restrictive say as our collective values become too godly, our collective values opt to no-longer uphold them.   When our individualism leads to the falling apart of society we raise the bar to create better virtues as it makes us more content, loved and at peace.  

Movement within the “stable compromise” domain has been explored many times but the Tytler cycle of history expands it very well.

 

In summary, a rules-based approach prescribes or describes in detail a set of rules and how to behave based on known and agreed principles. Whereas a principle-based approach develops principles which set the limits that enable controls, measures, procedures on how to achieve that outcome is left for each organisation to determine.

Risk frameworks help us to connect principles and rules

Having explored that a rules-based approach prescribes in detail the rules, methods, procedures, processes and tasks on how to behave and act, whereas a principle-based approach to creating outcomes crafts principles that frame boundaries, leaving the individual or organisation to determine its own interruption. 

In a linear system, we would agree on principles which would bound the rules.  

In a non-linear system, we would agree on the principles, which would bound the rules and as we learn from the rules we would refine the principles.  

In a complex adaptive system, we are changing principles, as our values change because of the rules which are continually be modified to cope with the response to the rules.

This post is titled “In a digital age, how can we reconnect values, principles and rules?” and the obvious reason is that rules change, values, which change principles that means our rules need to be updated. However, this process of learning and adoption depends on understanding the connection which offers closed-loop feedback.  An effective connection is our risk frameworks.


The diagram below places rules and principles at two extremes. As already explored we move from principles to rules but rarely go back to rethink our principles, principally because of the time.  Rules should refine and improve in real-time,  principles are generational.  However to create and refine rules we use and apply a risk framework.  The risk framework identifies risk and to help us manage it, we create rules that are capable of ensuring we get the right data/ information to be able to determine if we have control over risk.   As humans, we are not experts in always forecasting the unimagined and so when we implement rules things break and clever minds think how to bend, break or avoid them.  To that end we create more rules to manage exceptions.  However, occasionally we need to check that our rules are aligned to our principles and indeed go back and check and refine our principles. 

Starting from “Principles” these are anchored in ideas such as Human Dignity, Subsidiarity, Solidarity, Covenantal, Sustainability, The common good, Stewardship, Equality.  

Once we decide that one or more of these should anchor our principles and form a north star, a direction to travel in and towards. The reason to agree on the Principle(s) is that collectively we agree on a commitment to get to a better place. We state our principles as an ambition, goal, target with allow us to understand, manage and control uncertainty using a risk framework. The risk framework frame or bounds the risk we are prepared to take.  The risk framework enables us to define rules that get to our known outcomes.  We implement the rules to create controls using regulation, code and standards. Our risk frameworks use tools to identify, measure, manage, monitor and report on the risk, the delta in risk and compliance with the rules.  Whilst all is good we use the risk framework to create more rules and better framing and boundaries, creating better outcomes.  However, when the desired outcomes are not being created we revert to the principles, check our north star and take our new knowledge to refine/ redefine the risk we are prepared to take.

Data introduces new Principle problems! 

Having established this framework, the idea is to apply this to data.  We have an abundance of rules and regulations and as many opinions on what we are trying to achieve with data.  However, we don’t appear to have an agreed risk framework for data at any level, individual, company, society, national or global.  This is not a bill of rights, this is “what do we think is the north star for data and on what principle should data be?”  How do these principles help us agree on risks, and will our existing rules help or hinder us?

“what do we think is the north star for data and on what principle should data be?”  How do these principles help us agree on risks, and will our existing rules help or hinder us?

The question is how do our principles change when the underlying fabric of what is possible changes, the world we designed for was physical; it is now digital-first. Now we are becoming aware that the fabric has changed, where next?   By example, Lexis is the legal system and database.  With a case in mind, you use this tool to uncover previous judgments and specific cases to determine and inform your thinking.  However, this database is built on humans and physical first.  Any digital judgements in this database are still predicated on the old frameworks, what is its value when the very fabric of all those judgements changes.  Do we use it to slow us down and prevent adoption?  Time to unpack this

Physical-world first (framed as AD 00 to 2010)

Classic thinking (western capital civilisation philosophy) defined values and principles which have created policy, norms and rules.  Today’s policy is governed by people and processes. We have history to provide visibility over time and can call on millennia of thought, thinking and wisdom.  Depending on what is trending/ leading as a philosophy we create norms.  In a physical and human first world, we have multi-starting positioning. We can start with a market, followed by norms, followed by doctrine/ architecture - creating law and regulations  OR we can start with norms, followed by doctrine/ architecture, followed by market-creating law. 

Without our common and accepted belief our physical world would not work. Law, money, rights are not real, they are command and control schema with shared beliefs.  Our created norms are based on our experience with the belief.  We cope by managing our appetite to risk. 

Digital world first (frame as AD 2020 - AD MMMCCX )

People-in-companies rather than people-in-government form the new norms as companies have the capital to include how to avoid the rules and regulations.  The best companies are forming new rules to suit them. Companies have the users to mould the norms with the use of their data. Behaviour can be directed. Companies set their own rules.  Doctrine/architecture creates the market, forming norms, and the law protects those who control the market.  Policy can create rules but it has no idea how rules are implemented or governed as the companies make it complex and hide the data. There are few signs of visible “core” human values, indeed there are no shared and visible data principles.  We are heading to the unknown and unimagined.

The companies automate, the decisions become automated, the machine defines the rules and changes the risk model. We are heading to the unknown and unimagined as we have no data principles.

By example. Our news and media have changed models. The editor crafted control to meet the demand of an audience were willing to pay to have orchestrated content that they liked.  As advertising became important, content mirrored advertising preferences and editorial became the advertising and advertising the content.  Digital created clicks that drove a new model to anything that drives clicks works.  The fabric changed from physical to digital and in doing so we lost the principles and rules of the physical first world to a digital-first world that has not yet agreed on principles for data. 

Data is data

This article Data is Data explores what data is and is my reference to define data. 

Imagine looking at this framework of “principles, rules and risk” within the industry and sectors seeking to re-define, re-imagine and create ways for people to manage the digital representations of themselves with dignity.  How would say their data and privacy be presented?

With data (privacy, protection, use, collection) we have an abundance of rules and regulations and as many opinions on what we are trying to achieve.  We appear to be missing an agreed risk framework for individuals, company’s, societies (national &global)  

The stated GDPR principles are set out in Article 5

Lawfulness, fairness and transparency.

Purpose limitation.

Data minimisation.

Accuracy.

Storage limitation.

Integrity and confidentiality (security)

Accountability.

We know they are called “Principles” by the framing of the heading in Article 5, however, if we read them slowly are these principles, values or rules? Consider are these boundaries, stewardship ideals or a bit of a mashup.   By example to get round “Purpose Limitation,” terms and conditions  become as wide as possible so that all and or any use is possible.  Data minimisation is only possible if you know the data you want, which is rarely the case if you are a data platform.   If a principle of The European Union is to ensure the free “movement / mobility” of people, goods, services and capital within the Union (the ‘four freedoms’), does data identity ideals and GDPR align?  

Considering the issue about the “regulation of” Big Tech, in general should they exist, as no one entity should have that much power and control over people’s data and ability to transact? So the framings that accepts them as acceptable, won’t create rules that actually moves towards the principle of ending the current hegemony but rather just seek to regulate it as is.  If we add in open API’s and the increasing level of data mobility, portability and sharing whose “rules or principles” should be adopted?

How do your principles change when the underlying fabric of what is possible changes? The entire privacy framework, say in the US today, is based on early 1970’s reports written in the United States to address concerns over mass state databases that were proposed in the mid-late 1960’s  and the growing data broker industry that was sending people catalogues out of the blue. It doesn’t take account for the world we live in now where “everyone” has a little computer in their pocket.  Alas, IMHO, GDPR is not a lot better than rules with no truly human based core principles.

Conclusion

We appear to have outdated “principles” driving rules in a digital-first world. 

Our commercial world is now dominated by companies setting “their” norms without reference to any widely agreed-upon values. The down side of big tech gaining so much power that they are actually seen by people-in-government as “equivalent to nation-states” is telling.  Right now we need historians, anthropologists, ontologists, psychologists, data scientists and regular everyday people who are the users to be able to close the loop between the rules we have, the risk frameworks we manage and the principles that we should be aiming for.      

Take Away

How are we checking the rules we have are aligned to our principles?

How are we checking our principles?

Is our risk framework able to adapt to new principles and changes to rules?

How do we test the rules that define and constrain can create better outcomes?


rules - unpacking the word

In my post on Principles and Rules, I explored the connection between our human desire for principles, our commercial need for risk and our love of rules.  It explored the fact that we create rules, to manage risks, that end up not aligned with our principles and made some suggestion about how we can close the loop.  In the article, I skipped over the word “rules” without unpacking i

In my post on Principles and Rules, I explored the connection between our human desire for principles, our commercial need for risk and our love of rules.  It explored the fact that we create rules, to manage risks, that end up not aligned with our principles and made some suggestion about how we can close the loop. 

In the article, I skipped over the word “rules” without unpacking it.  This post is to unpack the word “rule”  The history and origin of the word “Rule” is here.  Irrespective of the correct use of the word “rule,” we use words in both correct and incorrect situations. Incorrect being there is a more precise or accurate word in the context or situation but we chose the word we do so as to create ambiguity, to avoid controversy, to soften the message and because of naivety.  We know that words and our language itself are filled with convenient generalisations that help us to explain ourselves whilst at the same time avoid the controversy created by unique circumstances. 

In the Principles and Rules article, the choice of the word rule was ambitious. This allows readers to apply their own context to it, thereby creating more relevance to their own circumstances when reading.  It was not a legal contract scenario, writing definitions at the beginning to provide that level of clarity and common interpretation. 

So in the idea was ambiguity - this post, however, is to expand an ontology of the word “rules.”

For me, you or someone;

Rules are written or unwritten or both

Rules are mine, created by me that you need to follow. They are yours, crafted by you that you need me to obey. They are shared and we believe that they create a better society

Rules can be the law, just a guide, the standard you need to meet or the rituals that creates success.  But which law, they one we should not break or the one where we follow the spirit?  As a guide to guide me from here to where.  As a standard is that absolute or is a range good enough.  My rituals, did I learn them, did you teach me or somehow are they just there?

Rules equally give you more freedom (safety, less murder) and remove your freedom (choice). Rules give me more agency and at the same time remove it.

Rules define my boundaries but are the ones I have created for myself and I have continually refined them as I learn, or are my rules ones that come from history; because we have always done it this way.  

Rules are they creating my view on values or are the rules I have someone else’s values?

Rules are only there to be broken

Rules allow me to create something as I have done something, have experience and have learnt. Rules allow me to repeat and not make the same mistake or improve and adapt.  Rules save me time and energy - I love my huristics

Rules allow me to manage, prevent and control risk

But whose rules are they?


The takeaway

When our principles become rules, do we question either the rules or principles enough?



Rebecca Rachmany

Road Trip in a Pandemic

When I got to Milano, I thought: How did they know? This feels just like a post-apocalyptic film. But how did the directors know this is how it feels? It’s not that surprising, of course. Cities have gone through wars and pestilence since the existence of cities. Milano after the plague, or not after the plague. Certainly the worst of it hasn’t happened yet. We are in the middle of it. Maybe

When I got to Milano, I thought: How did they know? This feels just like a post-apocalyptic film. But how did the directors know this is how it feels? It’s not that surprising, of course. Cities have gone through wars and pestilence since the existence of cities.

Milano after the plague, or not after the plague. Certainly the worst of it hasn’t happened yet. We are in the middle of it. Maybe just the beginning of it. Whatever it is.

I remember a warm data workshop with Nora Bates. She said: The disaster has already struck. It just hasn’t struck everyone yet.

Empty office buildings in Milan

When I left Ljubljana, why did I leave Ljubljana? Numbers were rising. Measures were tightening. Will I be able to get to Spain? Will I be able to get back? Maybe with my Slovenian plates nobody will stop me.

In Milan I sat with one of my father’s best friends in a rooftop restaurant in the outdoor seating. They took our temperatures before we got on the elevator. I can’t visit my father but at least his friend lives within driving distance of me.

Four hours is a long way to go out of one’s way for a meal and Milan doesn’t seem so advisable, but who knows when we will get to see our loved ones again? She gets me an AirBnB because hotels make me choke with the poisonous sanitizer they use to cleanse the air.

We have dinner. I tell her it’s hard to speak to Americans about what’s happening in the States. She says “Whenever I bring it up, your father says ‘I don’t want to talk about it.’.”. I don’t want to talk about it either, but it’s all I ever talk about. I’m American, or at least that’s what one of my passports says. My residency permit says Slovenia. It’s my only hope of staying out of a red zone at this point. Red zone. I’m not worried about disease. Civil war is another thing.

The laundry place wouldn’t take my underwear or socks. If I want laundry done, I have to do it myself, they said. I don’t know if that’s because of the pandemic or a Milanese tradition. At a little coastal town two hours East, the proprietor of the self-service laundromat did my laundry for me at no extra charge.

This is my first post on the Sufficiency Currency blog. It’s about the wisdom of taking a road trip in Europe in the middle of a pandemic. At the beginning of the collapse of civilization. In a bizarre moment of suspension, where things seem to be going on as usual; waiting for the other shoe to drop. America is in the throes of a civil war and California has no air and we’re sharing TikTok videos and Instagram sunsets. Everyone knows the economy’s dead, but we’re working and shopping and paying rent as if nothing happened, like we’re under hypnosis.

Or a psychedelic trip. One big global psychedelic trip, and not the good kind.

Who is that masked man?

Zipping along the highway in a rental car in the middle of the Zombie Apocalypse Psychedelic Bad Trip. That doesn’t sound very wise.

At one of the ecovillages, they make their own toothpaste. It’s not really toothpaste; it’s a kind of a powder. I asked one of the volunteers how to use the tooth powder. I knew the answer, but I was hoping I was wrong. She said: we just dip our toothbrushes into the jar. I don’t need a pandemic to know that dipping my toothbrush in the communal jar of tooth-powder is a bad idea. I brought my own toothpaste, like a civilized person. I brought the biological kind so they can’t complain.

As I drive my car along the Riviera, I see it out of the corner of my eye, for a brief second between tunnels, the dark blue sign with the circle of stars around the word France. If I hadn’t been paying attention, I wouldn’t have noticed. OK Google doesn’t say “Welcome to France.” When you cross between states in the US, Google tells you “Welcome to New Jersey.” In Europe, no such thing. OK Google is silent. She knows I crossed and I know I crossed. Maybe she’s pretending to honor GDPR as if she doesn’t know my every move. She knows if I am speeding but doesn’t tell the cops. She definitely knows that I got a small cut on my right index finger. Now she knows the prints of my middle finger, too. “OK, Google,” I say. “That’s me,” she answers.

The border crossing is easy. Finding lunch on the French Riviera in the off season during a pandemic isn’t. I have cake and coffee instead of something that feels French — or like lunch. The coffee shop has no WiFi. I guess you’re supposed to be enjoying yourself by the marina, not working.

The Sufficiency Currency project is about creating an alternative form of economic activity. Not marketplaces. Not money. An evolution of how we perceive our economic activity.

“People have always used money. What else is there?” people ask.

People haven’t always used money and there are still peoples on earth who don’t. Every system is born, lives and dies. Money is just a human invention and the financial system is like every other human invention. We can and will invent something else. Hopefully very soon.

A road trip to replace the world’s financial system in the middle of the Zombie Apocalypse Psychedelic Bad Trip.

Follow he Voice of Humanity project here.

Tuesday, 01. December 2020

Phil Windley's Technometria

The Self-Sovereign Internet of Things

Summary: Self-sovereign identity offers much more than just better ways to log in. The identity metasystem is really a sophisticated messaging system that is trustworthy, secure, and extensible. While decentralized identifiers and verifiable credentials have much to offer the Internet of Things (IoT), the secure messaging subsystem promises an IoT that goes well beyond those initial scenarios. Thi

Summary: Self-sovereign identity offers much more than just better ways to log in. The identity metasystem is really a sophisticated messaging system that is trustworthy, secure, and extensible. While decentralized identifiers and verifiable credentials have much to offer the Internet of Things (IoT), the secure messaging subsystem promises an IoT that goes well beyond those initial scenarios. This post gives and introduction to SSI and IoT. The follow-on post goes deeper into what a true Internet of Things founded on SSI can provide.

I've been contemplating a self-sovereign internet of things (SSIoT) for over a decade. An SSIoT is the only architecture which frees us from what I've called the CompuServe of Things. Unlike the CompuServe of Things, the SSIoT1 supports rich, peer-to-peer relationships between people, things, and their manufacturers.

In the CompuServe of Things, Alice's relationships with her things are intermediated by the company she bought them from as shown in Figure 1. Suppose, for example, she has a connected coffee grinder from Baratza.

Figure 1: Administrative relationships in today's CompuServe of Things (click to enlarge)

In this diagram, Alice uses Brataza's app on her mobile device to connect with Baratza's IoT cloud. She registers her coffee grinder, which only knows how to talk to Baratza's proprietary service API. Baratza intermediates all of Alice's interactions with her coffee grinder. If Baratza is offline, decides to stop supporting her grinder, goes out of business, or otherwise shuts down the service, Alice's coffee grinder becomes less useful and maybe stops working all together.

In an SSIoT, on the other hand, Alice has direct relationships with her things. In Operationalizing Digital Relationships, I showed a diagram where Alice has relationships with people and organizations. But I left out things because I hadn't yet provided a foundational discussion of DIDComm-enabled digital relationships that's necessary to really understand how SSI can transform IoT. Figure 2 is largely the same as the diagram in the post on operationalizing digital relationships with just a few changes: I've removed the ledger and key event logs to keep it from being too cluttered and I've added a thing: a Baratza coffee grinder2.

Figure 2: Alice has a relationship with her coffee grinder (click to enlarge)

In this diagram, the coffee grinder is a fully capable participant in Alice's relationship network. Alice has a DID-based relationship with the coffee grinder. She also has a relationship with the company who makes it, Baratza, as does the coffee grinder. Those last two are optional, but useful—and, importantly, fully under Alice's control.

DID Relationships for IoT

Let's focus on Alice, her coffee grinder, and Baratza to better understand the contrast between the CompuServe of Things and an SSIoT.

Figure 3: Alice's relationships with her coffee grinder and it's manufacturer (click to enlarge)

In Figure 3, rather than being intermediated by the coffee grinder's manufacturer, Alice has a direct, DID-based relationship with the coffee grinder. Both Alice and the coffee grinder have agents and wallets. Alice also has a DID-based relationship with Baratza which runs an enterprise agent. Alice is now the intermediary, interacting with her coffee grinder and Baratza as she see's fit.

Figure 3 also shows a DID-based relationship between the coffee grinder and Baratza. In an administrative CompuServe of Things, we might be concerned with the privacy of Alice's data. But in a Self-Sovereign Internet of Things, Alice controls the policies on that relationship and thus what is shared. She might, for example, authorize the coffee grinder to share diagnostic information when she needs service. She could also issue a credential to Baratza to allow them to service the grinder remotely, then revoke it when they're done.

The following sections describe three of many possible use cases for the Self-Sovereign Internet of Things.

Updating Firmware

One of the problems with the CompuServe of Things is securely updating device firmware. There are many different ways to approach secure firmware updates in the CompuServe of things—each manufacturer does it slightly differently. The SSIoT provides a standard way to know the firmware update is from the manufacturer and not a hacker.

Figure 4: Updating the firmware in Alice's coffee grinder (click to enlarge)

As shown in Figure 4, Baratza has written a public DID to the ledger. They can use that public DID to sign firmware updates. Baratza embedded their public DID in the coffee grinder when it was manufactured. The coffee grinder can resolve the DID to look up Baratza's current public key on the ledger and validate the signature. This ensures that the firmware package is from Baratza. And DIDs allow Baratza to rotate their keys as needed without invalidating the DIDs stored in the devices.

Of course, we could also solve this problem with digital certificates. So, this is really just table stakes. The advantage of using SSIoT for secure firmware updates instead of digital certificates is that if Baratza is using it for other things (see below), they get this for free without also having to support the certificate code in their products or pay for certificates.

Proving Ownership

Alice can prove she owns a particular model of coffee grinder using a verifiable credential.

Figure 5: Alice uses a credential to prove she owns the coffee grinder (click to enlarge)

Figure 5 shows how this could work. The coffee grinder's agent is running the Introduction protocol and has introduced Alice to Baratza. This allows her to form a relationship with Baratza that is more trustworthy because it came on an introduction from something she trusts.

Furthermore, Alice has received a credential from her coffee grinder stating that she is the owner. This is kind of like imprinting. While it may not be secure enough for some use cases, for things like a coffee grinder, it's probably secure enough. Once Alice has this credential, she can use it to prove she's the owner. The most obvious place would be at Baratza itself to receive support, rewards, or other benefits. But other places might be interested in seeing it as well: "Prove you own a Baratza coffee grinder and get $1 off your bag of beans."

Real Customer Service

We've all been in customer hell where we call a company, get put on hold, get ask a bunch of questions to validate who we are, have to recite serial numbers or model numbers to one agent, then another, and then lose the call and have to start all over again. Or been trapped in a seemingly endless IVR phone loop trying to even get to a human.

The DID-based relationship Alice has created with Baratza does away with that because DIDComm messaging creates a batphone-like experience wherein each participant knows they are communicating with the right party without the need for further authentication, reducing effort and increasing security. As a result, Alice has a trustworthy communication channel with Baratza that both parties can use to authenticate the other. Furthermore, as we saw in the last section, Alice can prove she's a bona fide customer.

But the ability of DIDComm messaging to support higher-level application protocols means that the experience can be much richer. Here's s simple example.

Figure 6: Alice uses a specialized wallet to manage the vendors of things she owns (click to enlarge)

In Figure 6, Alice has two coffee grinders. Let's further assume that Alice has a specialized wallet to interact with her things. Doc Searls has suggested we call it a "briefer" because it's more capable than a wallet. Alice's briefer does all the things her credential wallet can do, but also has a user interface for managing all the things she owns and the relationships she has with them3. Students in my lab at BYU have been working on a prototype of such an interface we call Manifold using agent-enabled digital twins called "picos".

Having two things manufactured by Baratza presents a problem when Alice wants to contact them because now she is the intermediary between the thing and its vendor. But if we flip that and let the thing be the intermediary, the problem is easily resolved. Now when Alice wants to contact Baratza, she clicks one button in her briefer and lets her coffee grinder intermediate the transaction. The grinder can interject relevant information into the conversation so Alice doesn't have to. Doc does a great job of describing why the "thing as conduit" model is so powerful in Market intelligence that flows both ways.

You'll recall from DIDComm and the Self-Sovereign Internet, that behind every wallet is one or more agents. Alice's briefer has an agent. And it has relationships with each of her things. Each of those has one or more agents. These agents are running an application protocol for vendor message routing. The protocol is using sub protocols that allow the grinder to act on Alice's behalf in customer support scenarios. You can imagine that CRM tools would be fitted out to understand these protocols as well.

There's at least one company working on this idea right now, HearRo. Vic Cooper, the CEO of HearRo recently told me:

Most communications happen in the context of a process. [Customers] have a vector that involves changing some state from A to B. "My thing is broken and I need it fixed." "I lost my thing and need to replace it." "I want a new thing and would like to pay for it but my card was declined." This is the story of the customer service call. To deliver the lowest effort interaction, we need to know this story. We need to know why they are calling. To add the story to our context we need to do two things: capture the intent and manage the state over time. SSI has one more super power that we can take advantage of to handle the why part of our interaction. We can use SSI to operationalize the relationships.

Operationalized relationships provide persistence and context. When we include the product itself in the conversation, we can build customer service applications that are low effort because the trustworthy connection can include not only the who, but also the what to provide a more complete story. We saw this in the example with two coffee grinders. Knowing automatically which grinder Alice needs service for is a simple bit of context, but one that reduces effort nonetheless.

Going further, the interaction itself can be a persistent object with it's own identity, and DID-based connections to the participants4. Now the customer and the company can bring tools to bear on the interaction. Others could be invited to join the interaction as necessary and the interaction itself now becomes a persistent nexus that evolves as the conversation does. I recently had a month long customer service interaction involving a few dozen calls with Schwab (don't ask). Most of the effort for me and them was reestablishing context over and over again. No CRM tool can provide that because it's entirely one-sided. Giving customers tools to operationalize customer relationships solves this problem.

A Self-Sovereign Internet of Things

The Sovrin Foundation has an IoT working group that recently released a whitepaper on Self-Sovereign Identity and IoT. In it you'll find a discussion of some problems with IoT and where SSI can help. The paper also has a section on the business value of SSI in IoT. The paper is primarily focused on how decentralized identifiers and verifiable credentials can support IoT. The last use case I offer above goes beyond those, primarily identity-centric, use cases by employing DIDComm messaging to ease the burden of getting support for a product.

In my next blog post, I'll extend that idea to discuss how SSI agents that understand DIDComm messages can support relationships and interactions not easily supported in the CompuServe of Things as well as play a bigger role in vendor relationship management. These and other scenarios can rescue us from an administrative, bureaucratic CompuServe of Things and create a generative IoT ecosystem that is truly internet-like.

Notes Some argue that since things can't be sovereign (see Appendix B of the Sovrin Glossary for a taxonomy of entities), they shouldn't be part of SSI. I take the selfish view that as a sovereign actor in the identity metasystem, I want my things to be part of that same ecosystem. Saying things are covered under the SSI umbrella doesn't imply they're sovereign, but merely says they are subject to the same overarching governance framework and use the same underlying protocols. In the next post on this subject, I'll make the case that even if things aren't sovereign, they should have an independent existence and identity from their owners and manufacturers. The choice of a coffee grinder is based simply on the fact that it was the example Doc Searls gave when we were having a discussion about this topic recently. This idea has been on my mind for a while. This post from 2013, Facebook for My Stuff discusses it in the "social" venacular of the day. The idea that a customer service interaction might itself be a participant in the SSIoT may cause some to shake their heads. But once we create a true internet of things, it's not just material, connected things that will be on it. The interaction object could have its own digital wallet, store credentials, and allow all participants to continue to interact with it over time, maintaining context, providing workflow, and serving as a record that everyone involved can access.

Photo Credit: Coffee Beans from JoseAlbaFotos (Pixabay license)

Tags: agents credentials decentralized+identifiers didcomm identity me2b ssi vrm iot picos customer+service

Monday, 30. November 2020

The Dingle Group

Guardianship in Self-Sovereign Identity

On Monday, November 23rd the Vienna Digital Identity Meetup* held its 17th event, the focus of this event was on Guardianship and SSI. Our presenter was Philippe Page of The Human Colossus Foundation and is a current member of the Sovrin Foundation Guardianship Working Group….

On Monday, November 23rd the Vienna Digital Identity Meetup* held its 17th event, the focus of this event was on Guardianship and SSI. Our presenter was Philippe Page of The Human Colossus Foundation and is a current member of the Sovrin Foundation Guardianship Working Group.

Guardianship is a legal status where one individual is under the legal care of another. A natural guardian relationship is between parent and child, legal guardians are persons who are recognized by the courts as having the legal authority and duty of care for another.

While there are long standing legal precedents and processes around the assignment, management and revocation of guardianships, these requirements were not met by existing digital identity management solutions. With SSI, this mechanism now exists. SSI can work in conjunction with traditional identity and credential management systems while being able to integrate into existing legal processes and provides a robust mechanism for revocation.

Guardianship is a complex topic, with many subtleties and layers. In the humanitarian sector the it is a reality of daily life when supporting and assisting migrants, refugees and displaced persons. It is a topic we all are faced with in our lifetimes; whether as a child (being cared for) or as an adult (caring for or being cared for). In this first event on this topic, Philippe has provided an overview of how SSI and Guardianship fit together and how SSI meets the lifecycle stages (Inception, Creation, Usage and Termination) of guardianship.

An objective of these events is to educate the community on how high assurance digital identities unlock new possibilities across all entities and industry sectors. Using the tripartite relationship in clinical drug trials of patient, doctor and pharma company, Philippe covered how guardianship can also be used to manage consent to derive a business value unlocked with high assurance digital identity.

Guardianship is an important legal concept, and will be a topic we will return to in 2021.

For a recording of the event please check out the link: https://vimeo.com/482803989

Time markers:

0:00:00 - Introduction

0:03:55 - Philippe Page - Introduction

0:08:03 - Overview

0:08:41 - Guardianship background

0:20:15 - Core Components

0:33:51 - Scaling and Standardization

0:39:04 - Bridging Economic Actors & Human Centricity

0:45:45 - Applications

1:02:08 - Questions

For more information on:

- Sovrin Foundation Guardianship Working Group : https://sovrin.org/guardianship/

- The Human Colossus Foundation : https://humancolossus.foundation/

And as a reminder, due to increased COVID-19 infections we are back to online only events. Hopefully we will be back to in person and online soon!

If interested in getting notifications of upcoming events please join the event group at: https://www.meetup.com/Vienna-Digital-Identity-Meetup/

*Vienna Digital Identity Meetup is hosted by The Dingle Group and is focused on educating business, societal, legal and technologists on the value that a high assurance digital identity creates by reducing risk and strengthening provenance. We meet on the 4th Monday of every month, in person (when permitted) in Vienna and online on Zoom. Connecting and educating across borders and oceans.

Sunday, 29. November 2020

Tim Bouma's Blog

The Power of a Secret

Photo by Michael Dziedzic on Unsplash Note: This post is the sole opinion of the author based on knowledge and experience gained at the time. The author recognizes that there may be errors and biases, and welcomes constructive feedback to correct or ameliorate. We all like secrets. When we possess a secret, it gives us a heightened sense of individuality — that we know some that nobody else k
Photo by Michael Dziedzic on Unsplash

Note: This post is the sole opinion of the author based on knowledge and experience gained at the time. The author recognizes that there may be errors and biases, and welcomes constructive feedback to correct or ameliorate.

We all like secrets. When we possess a secret, it gives us a heightened sense of individuality — that we know some that nobody else knows — giving us a special perspective or an option for the future that only we can exercise — in other words, power.

It turns out, imaginary or not, secrets are fundamental to the power that we have as individuals and institutions in the digital realm. Passwords, codes — those things that grant us or enable us to grant special access to those things that valuable, like bank accounts, emails, or the drafts and finals of our deliberations, the list goes on.

It turns out, that up until, August 1, 1977, secrets had a fundamental fault — we had to share them to use them. That meant you had to trust someone else, and that could eventually lead to the betrayal of your secret, and by extension, you.

In 1977, the public introduction of asymmetric cryptography heralded a new generation of secret capabilities. The first major capability was the establishment of shared secrets across insecure channels enabling encryption between two parties without the requirement of a secret backchannel. The second was enabling commitments using secrets that are not shared, more commonly known as digital signatures.

What had been discovered by Whitfield Diffie and Martin Hellman (and also Jame Ellis), is changing the world as we know it. It’s been only 43 years. Yes, that seems like an ice-age ago, but in the grand scheme of history, it is only a wink.

My concluding remark in this brief post is that you ain’t seen nothing yet (with apologies to BTO). I have been learning about many related schemes, based on that 1977 publicly-announced breakthrough: elliptic curves, homomorphic commitment schemes, proof-of-work, etc.

It’s one thing to understand these as mathematical, but it is another thing to understand what these things might be leveraged as institutional capabilities, either built by an institution itself or leveraged from an ecosystem that lets you keep your own secrets.

That’s the key — keeping your own secrets — keeping those things that give you the power.

Friday, 27. November 2020

MyDigitalFootprint

Creating Flow. Exploring lockdown audio lag and my exhaustion

So the technical term for that delay or lag from then you finish speaking to you hearing when the next person speaks is wrapped up in an idea of “Latency”.   Latency is measured in milliseconds (ms), which is thousandths of seconds. Latency for a face to face conversation is like zero. For say a landline call, it is defined by an ITU standard and is judged by the ability to offer a quality o
So the technical term for that delay or lag from then you finish speaking to you hearing when the next person speaks is wrapped up in an idea of “Latency”.   Latency is measured in milliseconds (ms), which is thousandths of seconds. Latency for a face to face conversation is like zero. For say a landline call, it is defined by an ITU standard and is judged by the ability to offer a quality of service.  Ideally, about 10ms will achieve the highest level of quality and feels familiar.  A latency of 20 ms is tremendous and is typical for a VoIP call as it is perfectly acceptable.  A latency of even 150 ms is, whilst noticeable, permitted, however, any higher delay or lag times and the quality diminishes very fast. At 300 ms or higher, latency becomes utterly unacceptable as a conversation becomes laboured, driven by interruptions and lack flow. 

We all know the phrases of “no-one left behind” or “you are only as strong as your weakest team member.” Well, the same applies for latency, one person in a remote place, low broadband speed, on a (shared) WIFI extension, with poor buffering on a cheap router;  we are now all down to the slowest person in the team. 

Analogy to get to the conclusion. 

“Jet lag”, also called “jet lag disorder,” is a temporary sleep problem that can affect anyone who quickly travels across multiple time zones. Your body has its own internal clock (circadian rhythms) that signals your body when to stay awake and when to sleep. Jet lag occurs because your body's clock is still synced to your original time zone, instead of to the time zone where you've travelled. The more time zones crossed, the more likely you are to experience jet lag. Jet lag can cause fatigue, an unwell feeling, difficulty staying alert and gastrointestinal problems. Jet lag is temporary, but it can significantly reduce your vacation or business travel comfort. Fortunately, there are steps you can take to help prevent or minimise jet lag.


Stay with me. We are bringing JetLag and voice/ video lag (Latency) together.  We know the effects of JetLag - fatigue, unwell feeling, loss of alertness, gastrointestinal problems and is temporary.  

The question is, can VoiceLag create the same. Anecdotally I believe Yes based on 8 months of video calls.  At the end of a day of video, Teams, Hangout or Zoom calls, we know we have fatigue, feeling unwell, loss of alertness, gastrointestinal problems and it is temporary. A good night of sleep, we can do it all again.   I know that now making a day of mobile or landline calls I don’t suffer the same.  

However, is this voice lag or voice latency or video time or a little part of each? We definitely know that video calls are exhausting, but the assumption for this feeling was the new structure, a new approach, differences and styles, watch yourself continually, only seeing one person, having to be present 100% of the time.  This is all true, but we also lack flow on video call due the latency and lag.  Lacking flow means conversation is paused, interrupted and slow. This delay takes a lot of energy.  We cannot get into flow to sharing our creative thinking, we have to hold ideas and opinions back, we have to wait for signals to speak - it is all exhausting.    

We need to focus on the remove of lag to create flow. We need to stop moving at the rate of the slowest person, let’s get everyone up to flow speed.

  




Wednesday, 25. November 2020

Aaron Parecki

GNAP Editors' Use of GitHub Issues

The editors met yesterday to discuss the issues that were pulled out of the previous draft text and document a process for how to resolve these and future issues. We would like to explain how we plan on using labels on GitHub issues to keep track of discussions and keep things moving.
The editors met yesterday to discuss the issues that were pulled out of the previous draft text and document a process for how to resolve these and future issues. We would like to explain how we plan on using labels on GitHub issues to keep track of discussions and keep things moving.

When there are substantive issues or pull requests, the editors will avoid merging or closing those outright, and instead mark them as "pending", so that these can be brought to the attention of the larger group. If no additional discussion happens on these, the merge or close action will be taken in 7 days. Note for this first round we are setting the deadline for the issues below as Dec 11th due to the US holiday and the fact that this is the first time using this process.

"Pending Merge"
When specific text is proposed in a PR (by anyone, not limited to the editors), and the editors believe this text reflects the consensus of the working group, this marks that the PR will be merged in 7 days unless there is a clear alternative proposal accepted by the working group.

"Pending Close"
When the editors believe an issue no longer needs discussion, we'll mark it "Pending Close". The issue will be closed in 7 days unless someone brings new information to the discussion. This tag is not applied to issues that will be closed by a specific pull request.

There are two additional labels we will use to flag issues to the group.

"Needs Text"
The editors suggest this issue needs additional text in the spec to clarify why this section is needed and under what circumstances. Without a concrete proposal of text to be included in the spec, this section will be removed in a future update.

"Postponed"
This issue can be reconsidered in the future with a more concrete discussion but is not targeted for immediate concrete changes to the spec text. When used on its own, this label does not indicate that an issue is targeted to be closed. An issue may also be marked "Pending Close", and this is used so that we can distinguish closed issues between discussions that have concluded or things that we may want to revisit in the future. Remember that closed issues are not deleted and their contents are still findable and readable, and that new issues can reference closed issues.

With these labels in mind, here are the list of issues and their statuses we were able to discuss on our last editor's call. The action on these pending issues will be taken on Dec 11th to give the group enough time to review this list. For this first round, many of the issues are marked "Pending Close" as we're looking for low hanging fruit to prune the list of issues down. In the future, you can expect to see more "Pending Merge" issues as we're bringing proposed text to review by the WG.

Postponed:

• Generic claim extension mechanism
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/131

Pending Merge:

• Make access token mandatory for continuation API calls
** https://github.com/ietf-wg-gnap/gnap-core-protocol/pull/129

Postponed and Pending Close:

• Fetchable Keys
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/47
• Including OpenID Connect Claims
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/64
• Application communication with back-end
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/82
• Additional post-interaction protocols
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/83

Pending Close:

• HTTP PUT vs POST for rotating access tokens
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/100
• Use of hash with unique callback URL
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/84
• Interaction considerations
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/81
• Expanding dynamic reference handles
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/76
• Post interaction callback nonce
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/73
• Unique callback URIs
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/55
• Instance identifier
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/46
• Requesting resources by reference
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/36
• Mapping resource references
** https://github.com/ietf-wg-gnap/gnap-core-protocol/issues/35

Tuesday, 24. November 2020

Matt Flynn: InfoSec | IAM

Modernization of Identity and Access Management

From the Oracle IAM blog: