World Music's DIVERSITY and Data Visualisation's EXPRESSIVE POWER collide. A galaxy of INTERACTIVE, SCORE-DRIVEN instrument model and theory tool animations is born. Entirely Graphical Toolset Supporting World Music Teaching & Learning Via Video Chat ◦ Paradigm Change ◦ Music Visualization Greenfield ◦ Crowd Funding In Ramp-Up ◦ Please Share

About Me

My photo
Autodidact. Laird o' the Windy Wa's. Serial Failure with Attitude. Bit of a Dreamer.
Andrew Douglas Hope 2016. Powered by Blogger.

Sunday, June 25, 2017

Cantillate

Graphical Toolset Supported Remote Or Distance Music Learning Via Video Chat


For better or worse, music teaching is migrating to the web. "Eye contact, body language, voice, pheromones and physical contact are not available on [social media]" - Susan Greenfield in "The Internet and 'Mind-Change'".

If on a personal level this exposes both teachers and learners to all the sensory dieback of the web, at a technical level it opens huge potential for innovative and supportive visual toolsets. Musicians hear songs when they read music, non-musicians seek visual patterns.

Online And Remote (Distance Or P2P) Music Learning

We can split music learning via the internet into two distinctive groups: for a very limited range of instruments online learning (generally pay-for-use video and audio collections, often backed up by written and almost exclusively static documentation but, increasingly, interactive notation).

For the vast majority of world music instruments 'online' is wishful thinking. Especially the world music instrument learner is often confronted with many lost hours chasing down basics such as fingering diagrams, score collections and example recordings. If interested in genre-authentic stylistic and fingering guidance, there is still little alternative to travel.

The most telling omission from simple online learning is, then, the remote, i.e. (more or less) live teacher. By 'remote' we mean all the visual immediacy (and slight latency) streamed video or (better) a video chat session provides.

So if we can make this distinction between online and remote, what are their practical impacts? The following diagram helps put things into context.

The most obvious difference between online and remote offerings lies in the teacher's ability to react to and accommodate user preferences.

Where an online learner may have had to hop between several online courses before hitting the mother lode, a live teacher can directly adapt at every level, accommodating particular instrument configurations (tunings and channel or scale lengths), the work of particular musicians, the tunes or styles the learner is particularly motivated to learn from, and of course provide direct encouragement when the going gets rough.

There is, however, more: remote teaching restores -at least in part- three critical sensory losses associated with the simpler online learning: eye contact, body language and voice. The learner senses the teacher's effortless skill, flexibility and enthusiasm, the teacher when the learner is dubious, confused or frustrated.

As we can see from the diagram, a switch between online and remote learning is possible at every point in the learning process. For all that flexibility, both learning modes suffer from the same major drawback: they can offer at best only extremely limited toolset support. This is where our aggregator platform steps in.

Adding A Graphical Toolset To The Constellation



On closer scrutiny, with instrument models, notation and theory tools available for any instrument, the problems previously challenging the casual autodidact are eliminated. Both autodidact and paying customer have access to the same online tools. The remote teacher can offer own musical and teaching skills, competing directly with video based courses. Let wealth flow. :-)

The cherry on this cake is that these tools are open source - and non-profit.

With an "Any instrument, any theory tool, any teacher or mentor, anywhere" mantra, it is set to democratize and dramatically strengthen understanding of comparative musicology, bringing remote teachers and learners together and providing entirely new meaning to immersive technical music visualization. This could, indeed, be the 'MySpace' users would have liked to have seen ... but simply never was.

Big, brave, open-source, non-profit, community-provisioned, cross-cultural and batshit crazy. → Like, share, back-link, pin, tweet and mail. Hashtags? For the crowdfunding: #VisualFutureOfMusic. For the future live platform: #WorldMusicInstrumentsAndTheory. Or simply register as a potential crowdfunder..

World Music Online Learning

So let's see if we can round these benefits up in one big firework of a diagram.



Potential Crowdfunder?

There are of course further hurdles to overcome, but once more widely understood (and financed), I suspect this project will prove unstoppable. I hope potential contributors and users for whom the concepts have already 'clicked' can be patient.

Conventional wisdom has it that new ideas be tested directly through crowdfunding. My feeling is that this project's full value will only become widely apparent through light but ongoing education. It has, indeed, taken many blog posts just to cover the basics.

Everything achieved to date has been concerned either with concept, proof-of-concept (demos & the associated videos) or this documentation.

Though simple in concept and relatively straightforward to test, the costs of a full implementation are too much to be borne by any single person. Hence the coming crowdfunding campaign, aimed at optimization and completion of a minimum viable product (MVP). The basic mechanisms have been prototyped and demonstrated - and work.

Implemented by a small number of scalar vector graphics, web and user experience experts, this will be centred around the first of what I hope will be many interactive, score-driven open source instrument model and theory tool contributions from the musical developer community. Fully graphical, online music lessons -and especially world music learning and music visualization- are a vast greenfield area of exploration, the potential benefits to humanity huge. Let's keep this accessible and non-profit.

Avidly Seeking Sponsors #VisualFutureOfMusic #WorldMusicInstrumentsAndTheory


If you can help with financing right now, please take a look at the current sponsorship options. Though well used to dismissing fears, with no income over a number of years now, I cheerfully admit to being in pretty desperate straights. :-)


Global Online Music Learning



World Music Distance Learning: A Galaxy Of Insights And Opportunities
World Music Aggregator Platform: Social Music And Dance #VisualFutureOfMusic #WorldMusicInstrumentsAndTheory Irish Folk Session-The Old Dubliner Hamburg 208-0075-f-hinnerk-ruemenapf-prev
Homemade Music, Wollaita, Ethiopia (15022080727) World Music Aggregator Platform: Ceili or Ceilidh Swing. #VisualFutureOfMusic #WorldMusicInstrumentsAndTheory Sape
Swindon Mela 2008, Town Gardens, Swindon - geograph.org.uk - 897414 Girl plays the tabla Oud class at Cairo's Beit el-Oud (House of Oud)


So what are the key differentiators in the online learning space? Here we compare traditional face-to-face teaching, a typical remote learning session using video chat, Soundslice's integrated (but bitmap-hobbled) approach, and our world music visualization aggregator platform's aims.




Keywords



online music learning,
online music lessons
distance music learning,
distance music lessons
remote music lessons,
remote music learning
p2p music lessons,
p2p music learning
music visualisation
music visualization
musical instrument models
interactive music instrument models
music theory tools
musical theory
p2p music interworking
p2p musical interworking
comparative musicology
ethnomusicology
world music
international music
folk music
traditional music
P2P musical interworking,
Peer-to-peer musical interworking
WebGL, Web3D,
WebVR, WebAR
Virtual Reality,
Augmented or Mixed Reality
Artificial Intelligence,
Machine Learning
Scalar Vector Graphics,
SVG
3D Cascading Style Sheets,
CSS3D
X3Dom,
XML3D


Read More

Wednesday, June 14, 2017

Cantillate

World Music Visualization Aggregator Platform: Key Features


We have the foundation for a music visualization (or visualisation) aggregator platform. What are it's key characteristics, and how do they benefit learners?

A number of platforms have begun to offer online (aka P2P, peer to peer, remote or distance) music lessons, but none so far have been able to offer a supporting toolset - providing a wide range of interactive, score-driven world music instrument models and music theory tools.

Remote Or Distance Music Learning

Those that succeed in bringing such tools to online world music teaching open the doors to highly interactive technical music visualization, an music-cultural avalanche, a revolution in comparative musicology, and will make a huge contribution to cultural understanding. Just as importantly, they will bring hitherto unknown freedom of choice.

Choice Of Notation Choice Of Instrument Choice Of Theory Tool
Choice Of Mentor Choice Of Supporting Tools Color Prefs etc


Peer To Peer Music Teaching And Learning Via Video Chat

In remote interworking between a teacher and learners, clearly simultaneous play will long remain a pipe dream, but a two-way musical conversation lies well within capabilities, offering the opportunity to listen, absorb, repeat - and, of course, question.

This dialog mimics the style of conventional (offline) musical workshops, where students are accustomed for example to exploring melody, fingering and chord progression alternatives, getting a tighter grip on the specifics of style and ornamentation, or absorbing a wide range of anchoring cultural information. These important and personal aspects of musical immersion are often ignored in many conventional online video courses.

The tools enrich this by on the one hand taking much of the effort out of preparation, and on the other generating an immersive learning environment and then driving it using the notation. The system can be considered "peer-to-peer" in that user environments are directly linked by shared controls. Notation, instrument model and theory tool behaviours on one end of the connection are shared with the others.



Ease of configuration and tight synchronization between the various forms media are central, but beyond that, our focus must be teacher, student, music (exercise), and instrument. Simplicity and transparency is all.

Judging by the lack of response to what has been documented so far, there is some scepticism to be overcome. I take heart, though, from the lessons from the past. :-)

A Galaxy Of Supporting Music Visualization Possibilities

If the core visualizations or animations are notation, instrument models and theory tools, secondary to these (we can imagine them as clustered around them) are supporting applications, handling, for example, teacher, student, band and other affiliations, and related planning or scheduling tools.

A third (and outer) zone of relevance would be that of related integrations - in which we can include standalone applications derived from core platform components. Here we might encounter integrations focussed on the impact of music on the psyche, esoterica such as music-astrological mappings, or simply personal 'Internet Of Things' preferences associated with specific learning environments or pieces of music.



With open source (commons-based peer-produced) models, the only limitation is the collective imagination. What one mind can visualise and articulate, others will be able to implement.

P2P Music Learning: Simple, Flexible, Personal

It's not rocket science. The development tools and libraries central to a comprehensive world music teaching platform such as this have for the most part long been proven. Scalar vector graphics (SVG) has, for example, been around for approaching two decades, and has long been supported in all major browsers.



The truly transformative link in our aggregator platform's technology chain is the visualization library D3.js, which provides a consistent and DOM-affine foundation for SVG manipulation.

With a surprising economy and elegance of expression, D3.js provides the means to dramatically augment the range of tools available to teachers and learners. D3.js tends to hide tedious detail (the 'how to do'), allowing developers to concentrate on tasks (the 'what to do'). It has a reputation for a steep learning curve, but this is dramatically eased with access to the code for established, open source instrument and theory tool models.

World Music Online Learning Platform: Simple Core Architecture

The DOM-based structures as shown in the banner image at the head of this blog are built around a few simple, interlocking mechanisms. These allow us, from raw data, to build static, on-screen models which can then be overlaid with fully dynamic, score-driven and synchronized animated graphics.

Taking no account of the mechanisms for instrument and theory tool exchange, we can view the system much as shown below. This view is trivial in the sense that it resembles that of a number of existing notation systems offering basic instrument (guitar, piano) display. Any similarities end in the detail, however, as in place of an Java / XSLT / XSL / XDom / XQuery coding constellation, we are working directly with javascript, D3.js and a couple of small helper libraries.



What really distinguishes the visualization aggregator platform as shown in the banner image are the mechanisms for personalization and comprehensive instrument model and theory tool exchange. Again there are clear structures and processes to these which dramatically extend the above model. In as far as useful to open source integration work, these will be revealed following successful crowd funding and initial release to the open source community.

Big, brave, open-source, non-profit, community-provisioned, cross-cultural and batshit crazy. → Like, share, back-link, pin, tweet and mail. Hashtags? For the crowdfunding: #VisualFutureOfMusic. For the future live platform: #WorldMusicInstrumentsAndTheory. Or simply register as a potential crowdfunder..

World Music Instrument Models - Diversity In Surfeit



With time, the platform can be expected to feature score-driven, interchangeable models of almost every world music instrument -of which there are certainly many thousands- and theory tools -of which there are hundreds- and allow straightforward behavioural comparison between them. How is this possible?

World Music Instrument Models: As Few Steps To Any Instrument Configuration

Using the 'lute' instrument family as an example, here we see how we might configure a Turkish bağlama or saz:

Modelling steps can be expected to mimic historical instrument construction and evolution. For simpler instrument such as those of the lute family in equal temperament (the default), configuration can be completed in as few as five steps. More complex instrument families may require a few more, but will always be manageable.

Complex wind instruments such as the modern flute, clarinet and saxophone can be expected to present among the greatest challenges, but manufacturers may be willing to assist. Our goal here is pragmatic visual modelling - and certainly not the fine tolerances associated with manufacturing.

World Music Instrument And Theory Tool Visualization: Social Value Wellspring

Each configuration can be saved for immediate, general use by the community. In this way, any generic instrument base can be used to generate hundreds of configuration variants subsequently available to the community via the classification trees (visual hierarchies) accessible only through the top-level menus row.



These entirely dynamic, interactive models represent a wellspring of social value in three distinct ways:
  1. used directly on the aggregator site, they provide a secure base toolset for remote P2P teaching by anyone
  2. embedded in an own, external site, they will allow users to build custom offerings, nevertheless around proven models
  3. because the models are to be open source and liberally licensed, they can be reused and adapted at will
The former will further understanding and community among musical cultures ('comparative musicology'), the latter two will allow anyone to develop and showcase own teaching content around specific instrument models. This might include entire instrumental courses, where even the simplest of exercises drives networked and synchronised understanding across animated notation, instruments and theory.

World Music Visualization Aggregator Platform - Avidly Seeking Sponsors #VisualFutureOfMusic #WorldMusicInstrumentsAndTheory

MusicXML: Serving The 2D And 3D Web

Potential Crowdfunder?

The exchange file format 'MusicXML' represents a piece of music as data in a serialised form suitable for streaming. In this sense each exchange file can be thought of as a rather wilfully structured database.

In contrast to a conventional database, though, to get 'useful' data (a melody line, the notes suggesting a chord), you are obliged to reconstruct the score and apply filters to it. Depending on the intended application, these may target (say) a score part associated with a specific instrument, a note selection at a particular position during playback, the scope of a loop, the tonal delta between two notes, or indeed the range of dynamics glyphs applied.

In this sense, the filters act as a form of database query, the score something of a query playground. Literally anything can be selected.

Note: there is another approach using XSLT stylesheets to query streaming MusicXML with XPath expressions. As this would be more difficult to steer and we are at no point dealing with actual streaming MusicXML, this approach is discarded.

The decisive factor is likely to be user convenience and freedom of choice: it is easier to handle filtering visually, on a reconstructed score. D3.js's selection mechanisms do just this - flexibly, precisely and concisely. Filtered data is then passed to dependent animations.

This data could also, however, be passed -via API- to other, external, applications.


The platform has the potential to exploit three transformational technologies: music visualization, artificial intelligence, and augmented or virtual reality. The above diagram attempts to illustrate use of MusicXML in two of these scenarios.

On the top left-hand side we have predominantly (but not exclusively) 2D music visualization, which are score-driven animations of music instrument models and theory tools built using SVG (scalar vector graphics) on the browser's DOM (domain object model). On the top right, we see how the same base musical data might be used to drive dramatic or gamified 3D narratives in a WebGL-based virtual world.

There are many possible configurations (directly via the API, using an embedded and interactive score display, or indeed app to app), but the central point is simply that any selections made on the notation could be shared with the 3D WebGL environment and hence fulfil considerably wider needs.

World Music Online Learning Speed And Focus

The brain is by all accounts a remarkably elastic organ. Given the right field of specialization and learning environment, a strong and free will and steady support, I have the impression we may all be capable of something approaching the extraordinary.

Our goal is as musically open and immersive an environment as possible, but with the live, virtuoso / teacher firmly at the centre. To that end, here an overview of some of the key musical qualities (key performance indicators or KPIs, if you will) aspiring musicians can be expected to hunger after, and which any learning environment should consciously support.

By structure, incidentally, the mental maps underlying instrumental navigation. Clearly, these can be literal or figurative.
With technical advance goes cultural erosion. We are left -on a purely technical level- dissatisfied with the media used in the direct teachings of earlier, culturally iconic players. In discarding the media, we discard what are often the last remnants of a culture.
a
Moreover, no single technology can replicate in all it's subtlety the directness of face-to-face teaching of a specific example of fine motor movement. This is why -despite technical barriers- it is so very essential that where we work online, it is with live, virtuoso musicians.

At the end of the day, we have feeds needing synchronised and merged to give our best shot at a semblance of immersion. Looking in a little more detail, we see how those feeds might flower into 2- and 3D animations across a whole range of technology stacks.

A Natural Step In The Move Towards Immersive Musical Learning

The traditional, face-to-face approach to teaching and learning mentioned at the bottom LHS of the following diagram embodied more or less everything needed to master an instrument over time. Indeed, in as much as there was also companionship and wide personal engagement, it also served social ends.

Until now online music learning has more or less just mimicked, step for step, that old mode of learning - if at distance. Technology will only really come into it's own when it breaks with conventional learning, and develops new, unimagined forms. How this happens -and whether, with diluted forms of contact, this proves a good thing- is anyone's guess.
ƒ


For the meantime, immersion in online music learning is technology-led, and any gain in immediacy simply a reflection of how it is applied. There is, however, always this sense of increasingly 'distancing' oneself from the original, living source. This makes me uneasy. Nevertheless, like it or not, technology is with us to stay, and so it is in our interest to make the best possible use of it that we can.

World Music Visualization Aggregator Platform: An End-To-End Value Chain

The musical data available at various points in the platform workflows are a resource to which both algorithm and machine intelligence can be applied, and which could be made available on-demand.



Online Music Learning: Liberate The Emotional, Delegate The Banal

Whether through soaring genre fluency, subtle contrasts in tempo, pitch, timbre or rhythm, or simply triggering each other's creativity through clear role punctuation, musical virtuosity is an act of profound empathy and intuition, of listening and reacting to other's play is ways that develop the emotional tension of the whole.

'Artificial intuition' is something of a hot topic in advanced computing, but is likely to take decades to find general application. Until then we have models such as the following for relationship between the emotional and the (for want of a better word) banal.

There is a threshold above which algorithm (and here I include artificial intelligence's pattern matching orgies) cede ground to simple human intuition and emotional intelligence. This is an area where creativity has free rein, and, naturally, where musicians love to hang out..

Driving Social Value: The 'Open Everything' Community Provisioning Stack

In the context of the web browser and DOM, there is a well-established (if perhaps hitherto not explicitly documented) open source resource pyramid providing a solid base for commons-based peer production. It lies in our hands to ensure that, for world music, the upper, as yet only marginally explored tiers are also brought into the realm of commons based peer production.

Our aggregator platform has the potential to provide the skeleton or framework upon which to integrate such services. How technologies such as artificial intelligence or virtual reality are integrated will at least be open to exploration and experiment.

World Music Visualization Aggregator Platform: Fully Configurable, From User Menus Down To Note Colours

The entire system has been conceived for personalization, both in terms of own (side) menu content, but also generally applicable conventions such as note colourings and fine settings such as tunings, custom fingering layouts and the use of aids such as capo.

Perhaps contrary to expectations, this is no rocket science, but simply reflects the power of modern data visualization libraries.

For all our technological advance, we are barely able to replicate millennia-old face-to-face (F2F) learning practices at distance - via video chat. Soundslice has set the bar high for synchonised video. Our aim is to go a little further, add as much value, immersion and immediacy to that dialog as current means allow.

World Music Online Learning: Cross-Cultural, Immersive, Global

Working with our hands seems to answer a fundamental human need - satisfied both in the high craftsmanship of instrument building, and in musicianship itself. With the inevitable arrival of a post-work society, I feel fairly sure both will enjoy a thoroughgoing renaissance.

Anyone familiar with the smell of freshly freshly shellacked wood, cured leather, machined synthetic ivory or the twang of a set of brass-wound steel strings will know just how satisfying a visit to an instrument builder's workshop or an instrumental workshop can be. It is our intention to catch and ride this wave, bringing what we can to the home, but more importantly, motivating people to get together.

Online Learning With Support For Any Music System (Notation), Instrument Or Theory Tool

The prize is sensational: the ability to configure the platform to your own needs, and then have the animations dance together -in tight synchronization- as the score plays.



The freedom to load one's own or a teachers immediate choice of exercise or score, have it drive an own, immediate selection of instrument models and theory tools, and the ability to visually follow and understand mappings between each.

Just as with the original, instrument models will be fine-configurable to fit own, immediate needs. Choose between different African Kora tunings, just bağlama or equal temperament oud, Venezuelan diatonic or classical chromatic harp, Bb, C or other clarinet (and particular historical form), A B- or C- system chromatic accordion, or simply between instruments of different scale lengths.

Visual Classification Of World Music Theory Tools

Going by Google Blogger's (anonymised) statistics, few visitors take the time to explore the Instrument and Theory Tool menus above, yet together with the notation, these form the core, if exchangeable, visual elements of the framework.

2 and 3D music-theoretical models normally represent some form of abstraction (for example reducing octave equivalents to some 'octave-agnostic' value) with the aim of exposing lean, underlying structure. Carefully aligned for manipulation by advanced, data-driven visualization libraries such as D3.js, though, data can be transformed and mapped in ways that we can barely imagine.

Mirroring Hornbostel-Sachs' visual classification for musical instruments, out aim is a simple, pragmatic classification system for theory tools - all of which, in the context of our aggregator platform, are all of course graphical in nature.

Online World Music Learning: Shared P2P Interworking Controls

Shared, P2P Interworking controls are foreseen on three levels>
  1. Configuration
  2. Environmental
  3. Playback


Configuration controls focus on the setting or sharing of instrument and theory tool definitions governing form and function.

Environmental controls govern the setting or sharing of preferences such as colour schemes, sound libraries and layouts.

Playback controls govern the speed, scope, and other behaviours of shared score playback.

Each control type could be applied across several peers. It would be up to the peers themselves to decide if they wish to retain these settings for future sessions.

World Music Visualization Platform: Value Added Core Strengths

The platform's core strengths are ripe for augmenation by two transformative technologies: artificial intelligens, and, by preference the Web3D forms of virtual and augmented reality.

The focus of the initial crowdfunding, however, is the browser DOM, as that is where this journey starts. Any expertise won there can be reapplied in other the other areas.

Music Visualization: A Foundation For Musical Storytelling

To date, timeline applications using D3.js or it's derivatives have been focussed on conventional data sets, but (analogously to our instrument and theory tool animations and using using either SVG or WebGL) we could equally well envisage stories being animated during score playback.



This suggests the merging of data supporting storyline development with that of data generated during score playback.

How 'creatives' develop this opportunity - and especially if it leads to new tools blending score and storyline - is something I'll be following with great interest.


Keywords



online music learning,
online music lessons
distance music learning,
distance music lessons
remote music lessons,
remote music learning
p2p music lessons,
p2p music learning
music visualisation
music visualization
musical instrument models
interactive music instrument models
music theory tools
musical theory
p2p music interworking
p2p musical interworking
comparative musicology
ethnomusicology
world music
international music
folk music
traditional music
P2P musical interworking,
Peer-to-peer musical interworking
WebGL, Web3D,
WebVR, WebAR
Virtual Reality,
Augmented or Mixed Reality
Artificial Intelligence,
Machine Learning
Scalar Vector Graphics,
SVG
3D Cascading Style Sheets,
CSS3D
X3Dom,
XML3D


Read More

Sunday, June 4, 2017

Cantillate

Online Music Education: Visualization Aggregator Platform Goals And Synergies


Currently, online teaching solutions are more or less stuck at the stage of video lesson collections - some offering synchronization with simple (for the main part single voice or part) notation, but most relying on a range of in-video inlays, and supporting only limited interaction.

The idea of an entirely graphical environment tying together notation, freely exchangeable instrument models and theory tools, and providing complete configuration freedom is perhaps the next major step in the development of online music teaching solutions.

Assuming such a system were available, what user needs would or could it answer? We identify some of the energies and synergies the world music visualization platform has the potential to release in the longer term.

'Why this list?'. Well: because all the points touched on have been mentioned in previous posts, but sometimes in rather specific contexts. I feel an urge to gather, summarize, abstract and generalize.

The list items have been loosely categorized as 'Emotional', 'Organizational' and 'Technical', broadly reflecting three interworking levels: that of users, tools and technologies.

Online Music Education: Visualization Aggregator Platform

Visual models are fine, but once in a while it can be helpful to give the imagination a clean slate. In contrast to earlier 'overview' posts this time without a swathe of accompanying images.

Complete Configuration Freedom Of Choice

Complete Configuration Freedom Of Choice
Scores and Notations Instruments and Tuning 2- And 3D Theory Tools
Supporting Applications Teachers / Mentors Color And Other Prefs

Goals and Synergies


Emotional Goals and Synergies

  • Counter the drift towards a global musical monoculture: celebrate instrumental diversity with world music instruments, tools, cultures and perspectives
  • Grass-roots, person-to-person teaching and enablement, connecting emotional intelligence, musicality and virtuosity
  • Driving global, cross-cultural social value: hence non-profit, largely open-source, with commons-based peer production (CBPP).
  • Center all activity around the user's choice of teacher and music
  • The re-democratization of music: encourage the playful, communicative and socially immersive aspects of music over the competitive
  • Exploit the brain's formidable visual and spatial reasoning
  • Fuel a renaissance in participatory community dance and music
  • Help render work irrelevant: providing occupation for idle hands and liberated minds
  • Data, story, goal, metaphor. Enable musical storytelling across a range of disciplines
  • Mutual empowerment through the gift of own time and skills
  • Support comparative musicology across a wide range of world musical cultures
  • Provide the tools to secure grassroots teacher earning potential - at lowest possible cost
  • provide an antidote to the music industry's more toxic business practices and role models
  • offer a familiar route into a musical future otherwise dominated by exponential technological advance and bewildering change

Organizational Goals and Synergies

  • Dynamic application of music theory as opposed to static explanation
  • Provide in as far as possible culture-agnostic visual models, leaving their interpretation to the teacher
  • Management of musical complexity: a catalyst for communication between musicologists, engineers and musicians
  • A toolset in support of P2P video chat, using ANY music source, instrument configuration, theory tool, mentor or location
  • Liberate musical learning and interworking through global visibility, ease of access and interactivity
  • An integration framework for artificial and machine intelligence with on-demand insights
  • Expose relationships between notation, instrument fingerings and abstract theory through score driven visual animation and interworking
  • Help users take instrumental and theoretical 'ownership' by giving them equal prominence with notation
  • Aggregate everything into one highly configurable and personalisable online music learning platform
  • Power of the crowd: from experiment to experience and consensus, with 'best of visual class' instrument and tool implementations
  • Reduce superfluous and toxic travel
  • Take a decisive stance against raptor economics and wealth hoarding. Get earnings working in and for the grass-roots music community.
  • Where visual classification trees are missing (as for theory tools), provide them
  • No advertising, but if there is a demand, links (via shopping-cart icon) to curated, user-rated directories of hand-built products
  • Provide sufficient but dependable, genre-authentic reference models

Technical Goals and Synergies

  • Focus: first and foremost graphical, thereafter -in as far as possible- multimedia-immersive
  • Simple, flexible and robust reuse mechanism favouring easily extendable visual model(s) over absolute dimensioning accuracy
  • Real-time single page application (SPA). Everything dynamically loaded: no installations, no 'plugins'
  • Build on other open-source initiatives in music publishing and accessibility
  • Event- and data-driven, in-browser SVG- and CSS using cutting-edge visualization libraries.
  • Device-responsive timeline-based but application agnostic -hence reusable- framework
  • Visual consistency through hardware timers, algorithmic placement and styling guidelines
  • Work to the 80:20 rule (get the important stuff right all the time rather than everything right some of the time).


Big, brave, open-source, non-profit, community-provisioned, cross-cultural and feather duster crazy. → Like, share, back-link, pin, tweet and mail. Hashtags? For the crowdfunding: #VisualFutureOfMusic. For the future live platform: #WorldMusicInstrumentsAndTheory. Or simply register as a potential crowdfunder..

Potential Crowdfunder?

Once crowd-funded, everything is going to go into overdrive. If you are reasonably familiar with the blog's content and can suggest areas you feel would add value, best to do so now. :-)

Because it's focus is entirely graphical, don't expect this platform to replace the many other excellent sites dedicated to mainstream instruments. Moreover, while providing profound visual insights, it leaves musical their interpretation (which may differ depending on culture or genre) to those who adopt it as a teaching platform, toolset and reference.

As with other music sites, subscription-based usage is envisaged. Our aim is to keep the costs as low as possible, and, as a non-profit, where surfeit exists, re.in.vest. There is nothing stopping a non-profits generating earnings: they just have to put it back to use - generally in the same year.

While the P2P interworking aspects have only been dealt with conceptually, the basic top-to-bottom mechanisms for instrument and theory tool configuration, population (personalization), docking and synchronization have been prototyped, demonstrate feasibility, and now beg to be fleshed out and scaled. Help us take delivery of social value to the next level.

What it will deliver is
o motivation (worldwide choice of teacher, instrument and tools)
o immediacy (cittern, CGDAD tuning, capo 2nd fret, driven by the user's own music source: no problem), and
o immersion (notation, instrumentation and theory tools woven into one coherent, synchronized dynamic whole - layered with stylistic cues from synchronized P2P audio and video).


Keywords



online music learning,
online music lessons
distance music learning,
distance music lessons
remote music lessons,
remote music learning
p2p music lessons,
p2p music learning
music visualisation
music visualization
musical instrument models
interactive music instrument models
music theory tools
musical theory
p2p music interworking
p2p musical interworking
comparative musicology
ethnomusicology
world music
international music
folk music
traditional music
P2P musical interworking,
Peer-to-peer musical interworking
WebGL, Web3D,
WebVR, WebAR
Virtual Reality,
Augmented or Mixed Reality
Artificial Intelligence,
Machine Learning
Scalar Vector Graphics,
SVG
3D Cascading Style Sheets,
CSS3D
X3Dom,
XML3D


Read More

Tuesday, May 9, 2017

Cantillate

World Music Visualization Aggregator Platform vs Soundslice


The world music visualization platform in focus here takes an approach quite different to that of Soundslice, perhaps the currently fastest-growing (solo-)instrument learning environment. A comparative review.

Many instrumental learners -especially those in heritage, folk, and traditional circles- get by learning from slowed audio. This mimics to a certain degree the face-to-face rote learning of earlier generations (and glorious accidents of memory likely responsible for much of the tune diversity).

Even so, alone the time overhead in finding, chopping up and slowing tunes can be daunting. This was, indeed, the motivation for 'Soundslice'. It can be thought of as a round-trip learning tool, in the sense that you start with an original recording, slow it down to a point where you can imitate it, then progressively speed up until playing along in perfect synchrony at the original speed.

Soundslice's focus is on simplifying life for the notation-oriented, 'learning-through-mimicry' instrumentalist, bringing a variety of learning sources into a single workflow - clear audio, simple, single-voice notation, a tiny selection of models of the most popular instruments, and synchronised video of 'virtuoso' (professional teacher's) play.

Soundslice takes a number of similar systems (such as InstantNotation or Knowtation) just a tick further in the drift towards on-demand immersive music environments.

Soundslice's primary achievements are: giving some visual context to what is essentially an audio feed; in time-synchronising the various other feeds with this audio.

A particular success are:
  1. the synchronization of video with external (as opposed to recorded, 'in-video') notation, and the notation's robust, well-synchronised looping controls. These provide both accurate positional and fingering clues, and very useful stylistic audio cues
  2. The reuse of fingering information gleaned from the various instrument-specific input file formats
It is difficult to overemphasise the value of the latter, but it immediately exposes one of the major weaknesses of current music exchange file formats: the inability to map to exchangeable, external fingering files. Even MusicXML, the W3C musical crown jewel, uses hard-coded, inline fingerings. In a world music context, this is cultural assassination.

Nevertheless, taken together, these to some degree plaster over one of the major shortcomings of notation - it's inability to adequately describe the subtler elements of style. Be it accenting, subtle rhythmic attack or delay, or the fine detail in ornamentation, these are all qualities notation is notoriously poor at conveying. Nevertheless, audio is far from unique to Soundslice. More on that below.
For all it's effectiveness, with the exception of looping controls, I haven't featured Soundslice's notation on this diagram, as there are many similar offerings elsewhere. (Had the notation been data-driven and the notes individually addressable/interrogable, I would have, as this opens whole new interaction possibilities). Moreover, Soundslice's approach more or less undermines the possibility of exploiting an area of huge and untapped potential: music visualisation.

What it does, though, it does extremely well, and at first glance, appears to answer pretty much all needs.

Today's needs are, however, not tomorrow's. We stand at the threshold to multiple technological paradigm changes:
  • music visualization
  • augmented and virtual reality
  • machine learning and artificial intelligence
These draw us inevitably from 'mobile first' towards 'data-driven', 'on-demand' (or, better still, 'need-of-the-moment'), immersive and ultimately, it seems, 'AI-first'. A comparison at this point seems both timely and interesting - on a number of levels.

Without distracting from Soundslice's successes, I'd like to focus in on a couple of opportunities it's design impedes - or even obstructs.

Big, brave, open-source, non-profit, community-provisioned, cross-cultural and batshit crazy. → Like, share, back-link, pin, tweet and mail. Hashtags? For the crowdfunding: #VisualFutureOfMusic. For the future live platform: #WorldMusicInstrumentsAndTheory. Or simply register as a potential crowdfunder..

Learning By Ear

Potential Crowdfunder?

Prior to that, however, I feel obliged to concede that for many older / 'traditional' players, even notation is overkill: slowed audio (mp3, more recently AAC) is widely accepted as entirely sufficient:
  • It mirrors the older (and much revered) practice of face-to-face learning around the hearth during long winter months.
  • It is well supported by 'slow-down-while-maintaining-pitch' tools
  • Audio is the fastest and shortest route to the brain. With our brains naturally (and with impressive accuracy) able to distinguish pitch, everything put in the way of ear learning simply slows down the mapping of audio via brain to instrument. Neural pathways take time and practice to form, yet folk virtuosos around the world are stunning testimony as to the power of learning by ear. Many don't read a note of music.
  • Hands-free play-along. Once a piece of music is set to automatic looping playback on an audio device, no further intervention is required. The mind is free to concentrate, the hand to play.
  • Location independent. With any good mp3/AAC player - down by the river, in a park, perhaps during a lunch break (effectively a mini-holiday, given the switch of brain hemispheres).
  • Learners develop a simple, intuitive sense of harmonic relations - applicable both in equal temperament and just intoned music cultures.
That may seem a daft admission coming from something of a visualization 'evangelist', but where audio supports musicality, music visualisation brings deep understanding. Poles apart.

Immediate Needs

There is a sense in which all current online offerings fail to deliver, and that is in satisfying immediate needs. While easy enough to find a modal scale, there may be no help with fingering on an instrument of your choice, or in viewing how it fits in with the broader modal landscape of the musical culture. Often, just to get at this information, you are obliged to scour online book & CD offerings, commit to an unnecessarily broad online course, or indeed follow something created for an entirely different instrument.

The counter-argument is that the brain develops according to the challenges we feed it. Accordingly, even a marginally immersive teaching environment should help develop multiple skills, in parallel, and across a broad front.

Yet notation's massive cognitive load has to be overcome before a note of actual music is played. This has clear impact on the speed at which we learn. Indeed, notation could be argued as having nothing to do with the actual music.

Younger players have grown with the internet, and so are perhaps more open to even slightly more immersive approaches. Soundslices's video and 'hardcoded' (non-configurable) instruments to some degree bridge the gap in providing limited visual orientation. These, however, remain a drop in the instrumental and music visualization ocean.

Soundslice In Detail

Business Model

Soundslice's focus is squarely on a limited range of music exchange formats -mainly those preferred by (gypsy swing) guitar players- with consequences throughout it's entire business alignment. Orchestral or even small ensemble music scores for example, are out.

Soundslice exploits a canny mix of franchising and subscription. In providing the tools needed to synchronise professionally-produced video with notation generated from a variety of exchange file formats, it not only drives own and other's (franchised) subscriptions, but transfers the responsibility for provisioning and promotion to the individual franchises. With a 30% commission on franchise sales, beyond the initial implementation for a given instrument, for it's owner a golden egg laying goose.

At the simplest level, Soundslice allows users to load their tune in a variety of (mainly guitar- and piano-oriented) formats and have it played back with synthesised audio. This is precise, if lacking in the fine musical tensions that distinguish a human player from a mechanical plonker. This does, however, provide for solid rhythmic accuracy - which for some beginners is perhaps no bad thing.

This form of usage allows display either in conventional music notation or so-called 'tablature', a form of pictorial shorthand specific mainly (but not quite entirely) to strung instruments. For this form of use, basic subscription rates apply.

So what help in expanding the platform's functional range is offered? Soundslice's Acceptable Use Policy states:
"The main goal of Soundslice is to help you learn to play specific pieces of music. It's primarily for educational use, and we hope our tools make you become a better musician.

Beyond that, Soundslice is rather open-ended, in that it allows arbitrary annotation of sound and video. We expect users will discover and invent many wonderful, unexpected ways to use it. Those are encouraged, as long as they're consistent with this policy".

This seems to hint some form of open / win-win further development strategy, but as it turns out: only as long as the subscriptions are paid. → This is a closed source, franchise model. Moreover, the most obvious area in which musicians can make a useful contribution is that of tablature glyphs for other melody instruments, and in providing fingerings. The former is reusable, the latter a unicum.

Technical Overview

Soundslice has been built using javascript in conjunction with recent HTML5 web multimedia elements. It reflects efficient and economic -if from a graphics point of view slightly dated- good practice, achieving not just responsive page elements but also responsive content.

Where a learner might earlier have worked with an audio recording, a page of ABC, tablature or a melody line and a few notes, Soundslice generates tablature and/or notation from one's own choice of (single-voiced) exchange file, some simple fingering diagrams and (for an extra fee) time-synchronised video and tablature/notation of some professional mentor's play.

It integrates and fully synchronises notation, video and simple instrument animation. It's appears to rely on tablature information for diagrammatic finger positioning.

All files necessary to full offline use can be retained in the browser's cache, making it useful for those times you are out of mobile or WIFI signal range.

As touched on above, for traditional or folk learners who have worked predominantly with slowed audio, this is already possibly too rich an offering.

For advanced folk musicians seeking a deeper insight into music-cultural structures and for notation-based learners interested in deeper aspects of harmony, it may prove too little.

Nevertheless, Soundslice occupies an odd niche, strangely isolated amongst technologies currently attracting a lot of attention.

In particular, the indications are that is neither 'live', nor 'real-time', nor strictly 'data-driven' - nor (curiously) does it rely on WebGL, which underpins much of the current work on WebAR and WebVR, (the web-based forms of augmented and virtual reality, collectively known as Web3D). Nevertheless, it is reasonably immersive.

How easy it would be to integrate elements of Soundslice into these virtual worlds is moot, but it's an entertaining thought..

It has a simple, intuitive interface and reacts quickly to modification. It gets round some problems inherent in it's implementation using simple but effective (creator's own word..) 'hacks'.

Though allowing Soundslice to do what it does very well, as we will see below, these will clearly hamper the product in it's further development.

Soundslice's Strategic Limitations

Several Soundslice design decisions bar it's further development into a comprehensive aggregator platform for all instruments and other animations. Let's try to summarise some of these.

Notation
  • Notation appears to be limited to the treble and bass clefs
  • Currently, piano is the most complex instrument (MusicXML 'Part') setting offered. Synchronization between left (bass) and right (treble) hand voices is possible only because -exceptionally- all the relevant information is present within this instrument's 'Part'.
  • With note selection apparently by offset rather than class or id, it could be technically challenging to extend the system to play multiple parts simultaneously.
  • The above limit harmonic exploration.
Audio And Other Libraries
  • Instrumental audio 'synthesis' is achieved not by generative techniques, but by calculating offsets into mp3 files containing fixed-length soundbites of given instruments at various pitches. A hack, but as it happens, a very effective one..
  • Because of the above, however, other than instrument choice there is no mechanism for selection of alternative audio libraries
Video
  • There is a distinct cognitive load associated with video: mentally mapping positions and fingerings from an opposing, dynamic instrument view. Our immediate need is for orientation and detailed positional information.
Animations
  • Because controls are time-offset-based rather than linked to specific on-screen notation elements, the scope for animation and/or P2P interworking is severely limited (there seem to be hints on the Soundslice site, however, that other interworking models may be in the pipeline).
  • Instrument displays are 'one-offs' - effectively hard-coded island solutions with no scope for nicities such as alternative tunings. In serving only a tiny range of already hugely popular instruments and attracting predominantly professional musicians into it's franchising model, Soundslice could be said to be feeding the trend to something of an instrumental monoculture. No doubt this will ease as a wider range of genres, instruments and styles are added.
  • Though likely to improve, the number of instruments for which tablature exists is very limited.
  • Instrument displays appear to be prescribed rather than user-selectable.
  • Tablature and instrument model fingerings and chord diagrams could be said to represent redundancy.
Technical

Soundslice creates it's notation in bitmap form using the simplest of graphical elements 'hard-coded' onto a DOM 'canvas' element. Once there, they are more or less fixed.
  1. In bitmap form, the visual mapping and transformation possibilities are severely limited. Where a bitmap is needed (for speed on a mobile device, or exported as an image to a virtual reality context), it could equally well be derived on-the-fly using SVG-to-image conversion on a server - at any size, and if desired transformed.
  2. Bitmapped notation tends to undermine the clearest motivation for parsing MusicXML using Javascript (to prepare notation glyphs for algorithmic placement, using javascript-based data visualization libraries).
  3. Given the above, there is no means of selecting own, preferred note colouring schemes
  4. Soundslice's focus is not on data, but on processing speed. It was built to the 'Mobile First' mantra. Tomorrows's immersive applications, however, will run on dramatically faster, high-resolution augmented and virtual reality devices, will be real-time, 'on-demand', data-driven, and in no sense pre-baked.
  5. The instrumental focus is on the popular, not the diverse. We are in an age of musical individualism, of experimentation, world music and self-expression. Yes, of group play - but preferably accommodating the exotic. Moreover, the platforms of tomorrow will connect real people learning from each other in social groups. If real-time play together over the internet is still a pipe-dream, on-line face-to-face teaching and learning are already being done. They just need the supporting toolset..

In sum, a tour-de-force in pragmatic scaling down for the mobile app market, but which undermines much of it's potential in multi-voiced and advanced educational environments. The following diagrams more or less sum it up.

Omitting any mention of virtual / augmented reality or machine intelligence, here we see how Soundslice, our Aggregator platform and the earlier and less sophisticated InstantNotation and Knowtation products are placed relative to each other in the move towards immersive online learning.

By diversity, we mean a wide range of world music instruments and theory tools animations. But why a 'dubious' model? Because animation diversity and platform speed are not necessarily in conflict with each other. It all depends how that diversity is achieved. More on that in a mo..

Meanwhile, as you may see:
  • Over time (and especially as devices and browser technology advances), Soundslice may quickly get left behind. It will, however, continue to serve well on legacy devices.
  • For Soundslice to achieve greater instrumental or music theory diversity, the current franchise model could lead to high organizational overhead.

Rationale For A Clear Alternative

While readily acknowledging that much of folk music is monaural, the following will allow us to work with all manner of score: from single voices through band works, choral pieces, chamber music, orchestral scores - and across many cultures:
  • the widespread availability of good traditional music scores, as collected by enthusiasts worldwide.
  • the appearance of powerful online notation editors such as Noteflight, which contribute to the wealth of open-source scores, and can be used to create exercises illuminating theoretical and harmonic principles.
  • new forms of microtonal notation.
  • mechanisms for finely differentiated instrument and theory tool configuration.
Our as yet unnamed aggregator platform focusses on notational, voicing, instrumental and theory tools diversity. I hope this post has demonstrated that it is in no sense modelled on the Soundslice approach, but on social value, data and the possibility of reuse across multiple technologies.

Coming back briefly to that 'dubious' model (see similar graphic earlier in this post), can we now put the aggregator platform's potential capabilities into perspective?
Music Visualization Aggregator platform vs Soundslice. #VisualFutureOfMusic #WorldMusicInstrumentAndTheory
This graph is attempting to tell us that through careful configuration of world music instruments, we can reduce the processing overheads involved in recovering and displaying any instrument to much the same level as those Soundslice requires to show one.

What is not perhaps immediately clear is that it will allow the user to populate menus with their own selection of instruments, theory tools and supporting applications, and that the automatic interworking mechanisms open up a galaxy of potential score-driven music visualizations. Another way of looking at things is in the context of the immediacy and immersion of the user experience. Over time, these have developed from (for immediacy) simple face-to-face learning through to the point where people are already learning (albeit with little tool support), P2P (peer-to-peer) across the internet, and for immersion from simple audio through to immersive Web3D. In that user's acceptance of new technologies is delayed, immediacy and immersion can be viewed as alternately 'leap-frogging' each other.
If we zoom in a little on the top RHS, we can perhaps see how Soundslice and our aspiring aggregator platform might fit in. In introducing synchronised video, Soundslice is leading in an era of on-demand, media-synchronised learning.

Nevertheless: 'on-demand' is not 'immediate-need-driven', and 'synchronised' is not 'real-time'. The central distinction is simple: whether or not media is synchronised with a live teacher. A peer-to-peer system linking live teacher and students with gamer-style controls would greatly simplify one-way synchronization - which, if not enough to allow play together, is certainly enough to teach.

Entirely automating synchronization of video stream with a running score during playback is, however, much easier said than done. This is an area where artificial intelligence can perhaps, one day, prove really useful. Until then (and even with the help of game protocol and back-end server tricks) any synchronization of a P2P teacher's play with notation on the student's end is likely to remain a simple, rule-of-thumb affair.

Pre-synchronised audio and video of the kind used in Soundslice is howeve perhaps not such a problem. Much of the groundwork has likely already been done for tools such as VLC (video) and Audacity (audio). Perhaps someone will take the time to investigate this soon: an open-source video (time) tokeniser program - similar to that apparently offered by Soundslice could be very useful all round.

Finally, our aggregator platform, in supporting music visualization in the widest sense and in sharing it's various feeds, bridges the data divide between the conventional browser DOM and the dramatic, gamified world of WebGL and Web3D.

To sum up, while more than enough for learning new tunes on a few mainstream instruments, viewed from the perspective of musical diversity, visual modelling and understanding, immediacy and immersion, there is abundant scope for advance. The next step is crowdfunding, for which, having read this far, I fervently hope I have your support. :-)

Finally finally, a blow-by-blow summary of the main differences between Soundslice and our anticipated aggregator platform:

Comparison between Soundslice and the Music Visualization Aggregator platform #VisualFutureOfMusic #WorldMusicInstrumentsAndTheory


Keywords



online music learning,
online music lessons
distance music learning,
distance music lessons
remote music lessons,
remote music learning
p2p music lessons,
p2p music learning
music visualisation
music visualization
musical instrument models
interactive music instrument models
music theory tools
musical theory
p2p music interworking
p2p musical interworking
comparative musicology
ethnomusicology
world music
international music
folk music
traditional music
P2P musical interworking,
Peer-to-peer musical interworking
WebGL, Web3D,
WebVR, WebAR
Virtual Reality,
Augmented or Mixed Reality
Artificial Intelligence,
Machine Learning
Scalar Vector Graphics,
SVG
3D Cascading Style Sheets,
CSS3D
X3Dom,
XML3D


Read More