World Music's DIVERSITY and Data Visualisation's EXPRESSIVE POWER collide. A galaxy of INTERACTIVE, SCORE-DRIVEN instrument model and theory tool animations is born. Entirely Graphical Toolset Supporting World Music Teaching & Learning Via Video Chat ◦ Paradigm Change ◦ Music Visualization Greenfield ◦ Crowd Funding In Ramp-Up ◦ Please Share

Thursday, March 9, 2017

Cantillate

Music Visualisation: Technical Details

From a creative freedom point of view, there are two important foundational elements in a world music visualisation platform such as described here.

Potential Crowdfunder?

These are that the core musical notation is 1) entirely data-driven and 2) scalar vector graphics (SVG) based.

With these in place, we have a technical foundation for -on the one hand- data manipulation, labelling and presentation freedoms, and on the other a host of transformations such as scaling, zooming, panning, rotation, flipping, reordering, colouring, mapping and visual linking.

With the web stack as context and visual creativity central to the whole concept, an end-to-end javascript-based approach is also implied. With few suitable music parsing libraries available and their scope often limited to western 12-tone equal temperament, much must be implemented more or less from scratch.

Here we give a brief glimpse of some of the guiding principles underlying the platform.

Despite a general dislike on my part mixing musical concept and technology, this post gets a little technical in places.

Big, brave, open-source, non-profit, community-provisioned, cross-cultural and gobsmacked crazy. → Like, share, back-link, pin, tweet and mail. Hashtags? For the crowdfunding: #VisualFutureOfMusic. For the future live platform: #WorldMusicInstrumentsAndTheory. Or simply register as a potential crowdfunder..


P2P Interworking



The platform in focus here will provide support for video chat teaching. This implies some form of interworking/control messaging, of which WebRTC (open source) or Firebase (proprietary and at cost) are candidates.

WebRTC is open-source, entirely free, is designed for use by web (as opposed to -say- VoIP) developers (familiar syntax), is supported natively in most browsers, is already being used by hundreds of vendors (including Facebook and Whatapp), and (though of questionable relevance to our visualization aggregator platform) even has some mobile support.

There are already a number of potential open-source toolkits available in this area, but a 'standout' feature among some is the concept of distributed WebRTC servers - hence with no single point of control or failure. An example of such an approach can be found at Matrix.org. More on basic P2P music communications as relates to our visualization aggregator platform >here<. Nevertheless, watch this space..


Geekspeak (Technical Delights)



Music Visualisation. Flexible Reuse Hierarchies. Implement Once. And Only Once. #VisualFutureOfMusic #WorldMusicInstrumentsAndTheoryFlexible Reuse Hierarchies
Implement Once. And Only Once.
Music Visualisation: On-Demand Loading, Dynamic Change. For The On-Demand Economy. #VisualFutureOfMusic #WorldMusicInstrumentsAndTheory
On-Demand Loading, Dynamic Change
For The On-Demand Economy
https://bost.ocks.org/mike/selection/ #VisualFutureOfMusic #WorldMusicInstrumentsAndTheoryConsequently Data-Driven
Unparalleled Presentation Freedoms


The impact of a data-driven approach on musical notation, instrument models & theory tools is profound. It opens a vast, subtle compendium of musical differentiations and transformations - which we can only hint at here. The best is undoubtably to come.


Music Visualisation: Features. An Overview. #VisualFutureOfMusic #WorldMusicInstrumentsAndTheoryFeatures
Some Achieved, Some Still Adrift
Music Visualisation: Leveraging Instrument & Tool Data Trees. #VisualFutureOfMusic #WorldMusicInstrumentsAndTheoryLeveraging Instrument & Tool Data Trees
Coding Examples? Post-Crowdfunding :-)
Music Visualisation: Slice & Dice The Data. With D3.js - Any Way Ya Want. #VisualFutureOfMusic #WorldMusicInstrumentsAndTheorySlice & Dice The Data
With D3.js - Any Way Ya Want



Simple Core Architecture

"The solution to complexity is abstraction, also known as information hiding. Abstraction is simply the removal of unnecessary detail. The idea is that to design a part of a complex system, you must identify what about that part others must know in order to design their parts, and what details you can hide. The part others must know is the abstraction". - cs.cornell.edu

The DOM-based structures as shown in the banner image at the head of this blog are at heart built around a few simple, interlocking mechanisms. These allow us, from raw data, to build static, on-screen models which can then be overlaid with fully dynamic, score-driven and synchronised animated graphics. Despite their visual complexity, with only few parts moving at any one time, processing overheads remain manageable.

There is nothing mysterious about the core architecture, indeed it is much like a number of other music platforms featuring simple interactive instruments such as piano keyboard or guitar fingerboard.

What really distinguishes the system, however, are on the one hand a consequent approach to musical property abstraction and reuse, and on the other, the mechanisms for personalisation: unhindered instrument model and theory tool selection and exchange. Though a substantial open source modeling phase will be needed to populate the platform with instruments and theory tools, there are clear underlying structures and processes. Here their (modeling) core:
The mechanisms for the interactive classification, discovery, selection, population, synchronization, sharing and person-to-person interworking will have to wait until after the crowdfunding. Patience..

Musical property reuse, on the other hand is so simple, it can be summed up in one diagram:

Key Achievements So Far

To round up this post, here some of the key technical problems overcome so far, using a 'red, amber, green' traffic light metaphor to represent progress. Left unresolved, any one of these areas would severely impede the platform's usefulness. Keep in mind these represent 'proof-of-concept' and nothing near production-ready states.



Basically, the further towards the top right hand corner of the diagram, the closer one is to an well-defined animation of real, practical use in supporting P2P interworking. Clearly the interworking controls are a topic of key future interest.

In as far as useful to open source integration work, further details will be revealed following successful crowdfunding and initial release.

Itching For Detail

For those with a slightly more persistent itch for detail, try this description of some of the technologies underpinning online gaming. In the long run, these are likely to feature prominently in our implementation.


Keywords



online music learning,
online music lessons
distance music learning,
distance music lessons
remote music lessons,
remote music learning
p2p music lessons,
p2p music learning
music visualisation
music visualization
musical instrument models
interactive music instrument models
music theory tools
musical theory
p2p music interworking
p2p musical interworking
comparative musicology
ethnomusicology
world music
international music
folk music
traditional music
P2P musical interworking,
Peer-to-peer musical interworking
WebGL, Web3D,
WebVR, WebAR
Virtual Reality,
Augmented or Mixed Reality
Artificial Intelligence,
Machine Learning
Scalar Vector Graphics,
SVG
3D Cascading Style Sheets,
CSS3D
X3Dom,
XML3D


Comments, questions and (especially) critique welcome.