Emilio Ros-Fábregas, IMF-CSIC

As part of the European project COST Action «A new ecosystem of early music studies» (EarlyMuse, CA21161), we organized at the Institución Milá y Fontanals of Research in Humanities (IMF-CSIC) in Barcelona a so-called Short-Term Scientific Mission (STSM) that brought together four prestigious scholars.1 During two weeks (10-21 June 2024), we presented digital projects, exchanging ideas about our respective projects and regarding a wide range of issues and tools concerning digital edition of music heritage preservation, annotation, and analysis.2 Owing to the diverse geographical and professional background and expertise of the four grantees invited to come to Barcelona, this experience was extremely useful to examine not only recent technological developments, but their possible applications taking into account also the different purposes, needs, uses, and contexts of the projects. We agreed in that it would be useful to share these ideas with a larger audience through this collaborative article in which each participant would be able to articulate his/her point of view; in the order in which their sections are presented here, they are: Kevin Page (University of Oxford, UK), Mark Saccomano (Paderborn University, Germany), Olivier Lartillot (University of Oslo, Norway), and Anastasiia Mazurenko (from Ukraine, but now working at the Slovenian Academy of Science and Arts, Ljubljana).3 See the participants in Figure 1.

The experience I have had with the creation and development since 2013 of the digital platform Books of Hispanic Polyphony IMF-CSIC (BHP: https://hispanicpolyphony.eu/), in open access since 2017, has been very useful to maintain a direct contact with the challenges of digital technology, as well as with the opportunities such platform offers to introduce students to the world of Digital Humanities and to develop collaborative musicological research through practicums and research stays at our institution.4 Thus, when the COST Action project opened a call for Short-Term Scientific Missions (STSM) for 2024, my colleague Andrea Puentes-Blanco and I proposed the organization of a STSM in Barcelona which would contribute to address challenges beyond our technological expertise and to examine the potential future development of BHP, with the following summary description:

Our proposal was in line with the general objectives of the COST Action, as well as with the specific «Grant Agreement Period Goals», in particular GAPG 4 (related to Working Group WG3—Publications): «Imagining innovative publication models to promote the dissemination of research in all its diversity (historical, performance, digital, etc.)»; see below the four contributions. We were able to fulfill also GAPG 3, related to Working Group WG2—Sources: «Describe ways of identifying and describing endangered music sources (including in Ukraine)», since one of the grantees is from Ukraine with direct knowledge about the problem; see below Mazurenko’s contribution. For the organizers of this STSM was also unexpected to find out that two of the four grantees, Lartillot and Mazurenko, had a particular interest in traditional music (of Norway and Ukraine, respectively), which connected very nicely with our project Fondo de Música Tradicional IMF-CSIC (FMT: https://musicatradicional.eu/home), now undertaking a complete renovation of its website.6

The discussions about the relationship between technology and music heritage in a broad sense, encompassing early music and oral tradition, were very fruitful.7 We agreed in that music heritage and all those involved with it, in one way or another, are in need of recognition as a European value. Moreover, it is important to understand also that digital technology should be approached and articulated to respond to different needs: for researchers interested in the data to be analyzed, on the one hand, and for the dissemination of musical heritage to a large audience, on the other; see below Kevin Page’s three recommendations. Likewise, as pointed out by Mark Saccomano, since sharing and reusing data is crucial in any digital scholarly project, «prioritizing the use of established standards and formats whenever possible… helps to maximize efforts to generate new knowledge»; this is presented in detail through his experience with the project Beethoven in the House. In addition to the tools for specific use in a project of the Norwegian Folk Music Archive, Olivier Lartillot proposes also a general computational music analysis framework, since, according to his hypothesis, «diverse musicological visions all rely on common cognitive mechanisms that can be adjusted through specific parameters…. this vision remains prospective, related to new developments in AI». Finally, Anastasiia Mazurenko reviews the present situation of Ukraine’s music heritage, at a time of war, placing ethnomusicological research in a rich historiographical context in order to explain its particular characteristics and present needs; for instance, some of the established technologies for written music in other European countries are not suitable, since the majority of recordings of Ukrainian folk music are not supported by staff notation.

In the following four sections, the presentations are based on very advanced specific projects and practical experience of their authors, which constitute a rare occasion for the non-expert reader to explore, in plain language, the latest developments and challenges of digital musicology related to the edition, annotation and analysis of music heritage. I thank them for their generosity, and leave as concluding remarks of this article Mazurenko’s conclusions, explicit enough about the sense of urgency concerning music heritage at risk of destruction.

Kevin Page, University of Oxford

In reflecting the state of the art for digital musicology, its publication and dissemination, one might observe that «systems» —by which it is meant here assemblies of sources, human expertise, and technologies— are often most successful and sustainable when they are designed and refined towards a clear and specific use. This is particularly the case for digital music editions, where any solution must pay close attention to the nature of the sources, their form and format, the need for analysis, visualization or presentation, and the audience to which any publication —in the broadest sense— is aimed.

Since our music cultural heritage is large, rich, and complex, there are naturally a multitude of uses to consider, including many already recognized and understood, alongside other anticipated as having great potential.8 Here we contend that it is not so much a question of identifying a single «correct» solution for all future systems,9 but rather a question of developing frameworks through which those creating new systems can distinguish and organize their needs, align them with viable technological affordances, and initiate the interdisciplinary dialogues which are required to realize future models of collaboration.

The following three recommendations, while by no means exhaustive, are offered as an initial framework through which we might begin to assess and structure such considerations.

1. New models of publication should reflect the iterative and incremental nature of music scholarship. As in any academic research there are multiple stages, which sometimes occur sequentially (but not always), sometimes repeat, and often feed knowledge back into preceding stages. Technologies exist to support each of these stages (sometimes several), however for the digital support for each stage to be most effective it should be selected to match requirements specific to that stage. In other words, a recognition that different stages of research have different needs, reflected in use. This is illustrated in Figure 2, which we approximate here to only three stages: sources, analysis, and publishing.

For example, during an exploration of sources a researcher will wish to search, identify, and refine items from music collections to take forward for analysis. The researcher is likely to desire the widest possible breadth of indexed content for discovery, alongside faceting by criteria they consider relevant (perhaps composer, period, or melodic patterns in incipits). Services supporting these needs are exemplified by RISM Online, DIAMM, the Internet Archive Live Music Archive and LUX; see Background section, below.

Whereas the data (derived from sources) and tools used in the analysis stage of research must be aligned with the specific scientific investigation being undertaken. Selected examples given below include the Delius Quartet Performance Annotation tool, and CALMA.

For publication (in the broadest sense), the priority is communication and dissemination to a particular audience, entailing a combination and potentially derivation of outputs from previous stages, made consistent and coherent through an editorial process. Selected examples given below include the Delius Catalogue of Works, and the Lohengrin Digital Companion.

2. New models of publication should reflect different needs when publishing «editions» versus the sharing of «data» —and potentially at intermediate points along this spectrum. By «edition» we broadly include all intellectually driven interpretations such as traditional academic formats (monographs, journals, etc.), but also curated websites. In comparison, by «data» we refer to the representations of underlying information upon which scholarship is performed.

For example, an edition designed for public engagement, such as the Lohengrin Digital Companion, has an interface organized around a strong narrative theme. Conversely, the CALMA dataset provides full access to all computational analyses of audio and tools to programmatically explore and analyze that data (but no predetermined graphical discovery interface).

These distinct needs require different technologies as appropriate solutions; where an ecosystem of technologies has been developed it can support the transition from data to edition(s). For example, International Image Interoperability Framework (IIIF) provides the underlying access to data publications; which in turn provides for the development of common visual explorers such as Mirador; mirador can be incorporated in general discovery interfaces such as Digital Bodleian; and also within research tools such as the BitH Annotator App. Through Quire, IIIF images are automatically transferred into digital editions.

We should also recognize the operational priorities of the individuals or institutions publishing data. A library catalogue will have been primarily designed and structured to support the operational needs of that library or special collection; when it is made available as data, the Application Programming Interface (API)11 should be transparent in offering the information «as it is». The availability of such data is preferable to no data —and more practical than offering a potentially infinite number of data editions curated to specific needs. Data publishing is thereby an enabler for more direct interaction between music cultural heritage collections and researchers, collections and communities (but not to the exclusion of editions).

For both editions and data, citation is crucial, yet the granularity will be different and should be considered alongside the format in which each is encoded. Other important considerations are those of data protection (ethical sharing of collections; copyright; attribution and credit), urgency (heritage at risk), and sustainability of service provision (minimizing complexity and costs where the service would be unsustainable if higher).

3. New models of publication should enable increased articulation and recognition of all contributory roles in the musical cultural heritage ecosystem. There is expertise and scholarship throughout musical cultural heritage. The skills and judgement of a cataloguer in a music library is equally crucial to the production of a future academic publication as the subsequent analysis of sources. In giving visibility of this value we better incentivize collaboration. Taking, as an example, a technologist who develops an algorithm to help segment ethnographic recordings into separate tunes which is a necessary and precursory step to symbolic analysis by a musicologist. Granting musicological recognition to the importance of this step aids the justification of this technological research, so supporting its ongoing funding. Similarly for the music librarian. Publication is a means to articulate and recognize this value. New publication models must therefore reflect and encapsulate this value in its representations of data and editions, throughout the iterative and incremental research lifecycle. Technologies can support this within publications. Ubiquitous application of globally unique identifiers to all contributions, not only sources, can enable their tracing and recognition. Common data and access standards (e.g. MEI, IIIF, Linked Data) often already incorporate facilities to this end within their specialized remits; these can be leveraged and connected.

In summary, the considerations above are the result of inspiration and elucidation arising from many interdisciplinary discussions with learned colleagues. In particular, reflecting upon the tools, projects, and scholarship included in the background below, alongside the other rich contributions described in the further sections of this article. Through such exchanges of experience and ideas, we might hope to move towards a more coherent lens by which we can understand existing digital infrastructure for, and around, music cultural heritage, access to it, and scholarship of it. In doing so, we aim to bring shape and direction to considerations for new digital efforts: in music publication platforms and interfaces; computational music analysis; and encoding and visualization of music, music analysis and audio. Plus, most crucially, the interactions between these elements which can be realized in the future.

Background context follows, with a focus on the personal research of the author of this section. The tools and projects here are complemented by those introduced in the remaining sections of this article and in the Appendix 1.

The Delius Catalogue of Works (DCW) is the first thematic catalogue of the composer’s works, and the first to be digital. It is a collaboration between the University of Oxford and the British Library, and uses the MerMEId software originally written by the Royal Library Denmark (for the Catalogue of Carl Nielsen’s Works). Notably, MerMEId provides a catalogue editing/authoring environment, and a separate publication platform, which create and export information according to the MEI Metadata module. This illustrates the additional flexibility of MEI for applications beyond notation.

MELD (Music Encoding and Linked Data) is a flexible software platform suitable for creating new forms of «publication», developed at the University of Oxford e-Research Centre and since used by several academic research projects to combine digital representations of music (such as, but not limited to, audio and notation) with contextual and interpretive knowledge in the Semantic Web. Technically, it combines use of MEI XML and the Web Annotation data model (in JSON-LD), often with the SVG XML output of Verovio and IIIF compatible image sources. Combining such clearly scoped standards, each refined to a particular purpose, through a consistent and extensible framework enables the assembly of many different «apps» using some (but not necessarily all) of these technologies. The breadth of possible MELD uses can be seen in apps variously supporting: the (manual) annotation of a live string quartet performance,12 with the captured resources reused in a «Discovering Music» web article; a bespoke «super-instrument» incorporating MELD alongside other music technologies for a new interactive composition and its performance;13 and the Lohengrin Digital Companion which is a novel interactive «time machine» to accompany an academic essay and explanatory video.14

Linked Art was introduced as a new standard under development to share collections data describing art through a common model and API, intended as a complementary technology to IIIF, and following the community model which has made the IIIF standards so successful. LUX is a compelling example of a unified discovery system, built using Linked Art, across the collections of Yale University.

CALMA (Computational Analysis of the Live Music Archive)15 was an earlier approach to the automated analysis and annotation of audio data in the Internet Archive Live Music Archive. It embodies a «layered digital library» approach, in which different users or institutions can create and (if they wish) publish data and results, which are interconnected with each other and the underlying collection sources. This approach informed, and has been carried forward and refined, in research using the MELD, and including Beethoven in the House.

Mark Saccomano, University of Paderborn

In order to provide an example of how several digital standards, digital tools, and digital sources can be brought together to support a fully digital musicology research project, the EarlyMuse Short-Term Scientific Mission in Barcelona featured a detailed presentation of the Annotator App, a product of a collaborative UK-German-Swiss research project titled Beethoven in the House.16 This section will describe the tool itself and its aims, and how it might be able to address some of the current issues in the preservation and digitization of historical music sources discussed during the sessions. It also recounts some of the difficulties encountered while developing the app and how these issues highlight some considerations to keep in mind when using experimental research as a model when embarking on a digital musicology project.

The Annotator App was developed to access and compare remote digital resources and assist in performing routine musicological research tasks such as browsing collections, assembling work sets, and comparing and annotating sources. The app is a combination of established, standardized methods for encoding and exchanging music data, advanced technology in music encoding and music information retrieval, and specially developed, project-based tools that enable manipulation and annotation of digital materials. Research reports released by the project not only explained the project team’s development efforts, but were also intended as an illustration of the complex layers of structure at work beneath the application's interface —and how those layers contributed to the interoperability and sustainability of the project’s research output.

At the core of the application is a suite of independent tools for creating and working with digitized music sources: MEI (Music Encoding Initiative), Verovio, MELD, Cartographer, and Edirom. MEI is a widely used standard in libraries and archives for music encoding;17 MELD (Music Encoding and Linked Data) is a framework that integrates MEI with Linked Data, allowing elements of an encoding to be enriched with data from the web;18 Cartographer is a music information retrieval tool developed by the Edirom group that uses a measure detector to recognize and encode barlines from a printed score;19 and Verovio is an online toolkit that can render a notated score from an encoding and display it as a graphic image in a web browser.20 One of the many advantages of using MEI is that it is a widely used standard for making digital music editions, allowing researchers to form work sets from any of a number of public MEI repositories. In addition, its elements are uniquely addressable, allowing granular access to the musical data. MEI can also be extended and specialized, making it suitable for application beyond music editions, such as music analysis and music information retrieval. Along with MEI, the Annotator App takes advantage of IIIF (International Image Interoperability Format), a standard for the delivery of images over the web.21 As is the case with MEI, when institutions publish images using the IIIF standard, their resources can be accessed, shared, and integrated more easily across different platforms and with a variety of tools.

To make sure that users were not limited to selecting and commenting on strictly notational elements in a score (such as notes and measures), a new and flexible model was created, one that allowed musical structures to be targeted at various levels of scale and at various levels of abstraction (such as motives and themes).22 The model was expressed as Linked Data, which greatly enhanced the findability and shareability of the project's research objects: selected musical objects and passages, along with their associated annotations.

Thus, the Annotator App exemplifies a repeated theme of the Barcelona STSM: prioritizing the use of established standards and formats whenever possible. The benefits of using widely accepted standards are manifold. Chief among these is the possibility for sharing digital resources, including the products of research. Shareability and reusability of data are important considerations in any digital scholarly project and help to maximize efforts to generate new knowledge. Because of the wide adoption of standard formats like MEI and IIIF, there is ample support for their use, making them appealing choices for smaller scale projects with limited resources.

The project’s annotation structure also accommodated the formal inscription of the workflow and software utilized to arrive at a study's claims and conclusions.23 Not only are the sources and selections integrated into the research output as Linked Data, but the processing procedure itself is likewise encoded, so it can be replicated, verified, and, if desired, extended. When structured in this manner, evidence for musicological claims can be easily dropped into reports and articles, as well as blogs and correspondence, and easily accessed by users with widely available tools for viewing text, images, and music encodings.

The integration of datasets in this manner was a topic that came up often in STSM discussions. It is easy to imagine a future where music resources can be instantly located and —through a shared vocabulary of metadata terms, instituted across all known digital collections, held by any cultural institution— displayed with links to encoded music files, digital scans of manuscripts and scores, variants in different collections and different editions, plus analyses, recordings, arrangements, secondary literature, and performance history. Linked Data is often considered a means to achieve this goal, and is thus an exciting topic for researchers interested in gaining access to obscure or physically remote sources, or simply in assembling large datasets for analysis.

This vision of a future musicology was a fundamental impetus behind the Beethoven in the House project, and an aspect of the project that might be missed at first glance. Researchers wanted not only to formulate practices that would assist in the preservation and access of data and analysis, but also to contribute to the realization of an international, interoperable framework for the preservation and access of research artifacts at every stage of the research cycle. These are ambitious goals, and when a fledgling initiative is seeking guidance for infrastructure development and best practices, it is wise to try to tease out some of these underlying goals when seeking appropriate solutions from project reports. For smaller, independent endeavors, a careful consideration of costs and benefits of implementing technologies such as Linked Data is warranted.

An alternative to implementing state of the art tools is to adhere to the principles of Minimal Computing, which recommend matching tools not only to the job at hand, but also to the resources at hand.24 Keeping technical requirements to a minimum increases usability among users with less direct access to technology and know-how while helping avoid unnecessary overhead and maintenance costs associated with more complex systems. An additional benefit to this approach is enhanced sustainability. All digital music projects have maintenance costs associated with the technology used to deploy them. The fewer resources a project requires, the better chance it will have for long term survival and lasting impact. For example, some digital projects can consist of a digital collection or online exhibit which may be suitable for hosting on a readily available and relatively simple to use platform such as GitHub. An example of this is the Rebalancing the Music Canon website, which uses GitHub to both manage its workflow and to display the contents of its collection.25 Similarly, Quire is a low maintenance option for publishing materials on a static site that is optimized for displaying images.26 These Minimal Computing solutions can be especially useful when there is an urgency to the preservation of materials in at-risk collections.

Another important consideration when reviewing project reports is the use of dependent technologies, as these can seriously impede progress in any digital project. For the Beethoven in the House project —with its goal of setting up an all-digital, all Linked Data workflow— some of the dependencies and platforms used were of necessity leading-edge technologies. Reliance on new tools and frameworks, however, can lead to significant challenges for both implementation and reuse of the software, as became evident in the Beethoven in the House project. For storing and sharing annotations, the project's Annotator App relied on a protocol for a distributed storage platform called Solid, and a free community-supported service provider that hosted cloud-based storage containers called Solid Pods.27 These Solid Pods were chosen by the developers primarily because they offered a means for users to host and selectively share the annotations they created with the app, while also operating in an entirely Linked Data environment. The broader appeal becomes apparent, though, when viewed in contexts relevant to historical collections (and that were noted during the STSM sessions): namely, for archive repatriation —where digital copies can be made available as an interim solution for communities without the resources to house materials— and in assembling local heritage collections from resources dispersed in different institutions and different countries —an effort that would require different access levels in order to be practicable for stakeholders, as well as useful to different user groups (e.g. performers, listeners, researchers, etc.).

Community software, however, is only as strong as the community that supports it, and throughout the life of Beethoven in the House and afterwards, activity in the Solid community at large steadily decreased. As a result, there was not a strong developer network that could be consulted for answers to operational questions, nor did the community seem to make much progress developing and stabilizing the platform or the protocol. This ultimately made Solid Pods less than ideal for the continued use of the project software and its associated workflow, although no one could have foreseen this at the time the project began.

Importantly, however, the pods did not distort the data they handled. The data was still expressed as RDF statements that could be exported as a JSON file, a standard format for data interchange on the web. This is perhaps the most important consideration when it comes to research data management: the use of standard (hence, sustainable) formats. No system should bind users to a particular framework in order to use its data. So, in the end, it did not «cost» the project anything to test out these pods, while their use by the developers could still serve to promote the notion of decentralized storage with customizable access controls.

The successes and difficulties experienced by Beethoven in the House are typical of any research software project, and their lessons are part of the research findings. Fortunately, the existence of professional networks among participants in the project meant that outside expertise could be drawn upon to solve some of the implementation problems. Those same networks also helped to disseminate software and methods that could be useful for other research teams. The existence of such collegial networks, while often taken for granted, are crucial for the success of any scholar or team. Not only can they contribute experience and expertise that are not available in published reports and documentation —making for more efficient progress and more reliable software— they can also help ensure the widest reach and greatest utility of the project’s research and resources.

As is clear from the foregoing, some of these tools and techniques do not easily combine with one another —Minimal Computing and Linked Data, for example. Even though the general concept is easy to grasp, creating records in Linked Data can be exceedingly difficult to implement. It requires an understanding of how a dataset can be expressed as RDF (Resource Description Framework) —the Subject-Predicate-Object model that is used in Linked Data to identify and represent relationships among elements. It also requires sufficient knowledge of semantic web initiatives to create or adapt data models for the kinds of objects being cataloged. Ideally, this should be done with technicians and stakeholders who have an overview of what models are available and which identifiers have wide purchase in the field. And because of phenomena like «data friction» —the inevitable difficulties that impede the smooth transfer of data between formats, systems, and institutions— scholars skilled in digital librarianship are often needed as well.28 Sometimes, however, it may be enough to have trusted, knowledgeable colleagues to turn to for suggestions and advice, although the availability of such contacts should be ensured before embarking on any project concerning cultural heritage materials.

This informal cooperation, in fact, has been identified as the antidote to data friction.29 Knowledge sharing is also recognized as a key component in the successful integration of data among cultural heritage institutions.30 And the point was made clear in all of the stories, presentations and discussions throughout the STSM in Barcelona: supportive relationships among colleagues with specialist knowledge, willing to share their experiences with others, are an invaluable resource. The field of digital musicology is fortunate in that it is filled with practitioners who are generous with sharing their expertise, and it benefits as well from large, formal initiatives such as the EarlyMuse COST action which works to actively cultivate these networks.

Olivier Lartillot, University of Oslo

My approach to computational music analysis encompasses both audio and score-based methods. The approach, aimed towards a «Comprehensive Modelling Framework for Computational Music Transcription and Analysis», seeks to automate music analysis across a wide range of dimensions, using either score or audio (through automated transcription). One goal is to develop computational tools to preserve and enhance the cultural significance of music repertoires, with a present focus on Norwegian folk music. This is being done through a cooperative project with the National Library of Norway related to their collection of Norwegian folk music. This collection, funded in 1951, includes audio recordings of solo performances on various instruments (such as the normal and Hardanger fiddle, langeleik, guimbard, harmonica, clarinet and various flutes or psalmodicon), as well as vocal performances, in the form of singing and joik. By 1970, it contained 1,327 reel-to-reel tapes corresponding to 750 hours of music.

As part of the collaboration with the National Library in Oslo, the project initially focused on digitizating the tape recordings, concentrating on a specific collection of 343 tapes that are over 50 years old, due to copyright considerations. The challenge was to transform unstructured audio tapes into a systematic dataset of melodies, resulting in 20,000 pieces of music processed. To achieve this, we developed a specific tool called AudioSegmentor.31 The segmentation process is semi-automated: first, we use a Sound Analysis tool developed by Apple32 that provides a series of probability scores for various general sound classes (e.g. «Speech», «Singing», «Music», «Violin») for each successive second of the tape. This data is not sufficient to clearly determine where each tune starts, so we designed a rule-based system to estimate segmentation points from the data. However, the automated tool does not always work perfectly, so we also developed a graphical user interface that allows the operator to verify and correct the estimated segmentation points. The interface also allows reassignment of tune metadata from the original database to the corresponding segmented tunes.

This tape segmentation has enabled the individual tune recordings to be made available online in the form of a digital catalogue, hosted by OSF (the Open Science Framework).33 Additional metadata and links to metadata authorities, such as VIAF (Virtual International Authority File) and Wikidata, are under consideration. Further work is planned to ensure the accessibility and longevity of the digital catalogue through efficient data management and linking with other catalogues.

Besides initial tape transcription, the core activity as part of this collaboration involves automatically transcribing the audio recordings into scores. The focus has been on the music played on the Hardanger fiddle.34 The first step involves detecting each note being played, including the temporal position of note start and end in seconds and the pitch height. For contemporary playing of the Hardanger fiddle, where each note has a stable pitch height, we can extend state-of-the-art AI approaches based on machine learning. To train the AI model, we hired professional musicians to record and annotate their performances using a new annotation software called Annotemus.35 This software ensures precise annotation through adapted visualization and sonification. The resulting AI solution demonstrates very satisfying capabilities for accurately detecting notes, timing, and pitch height. However, we are now testing how well this approach works for older recordings. For music where notes do not have stable pitch height, current AI approaches may be insufficient, and other methods based on signal processing and cognitive modelling are being developed. We are turning our research in music transcription into a service for music professional, especially in music archives and in music production, called muScribe.36

The subsequent task is to detect the underlying beat to allow rhythmical transcription. For music with a regular beat, an AI tool such as Madmom37 can be used. However, Hardanger fiddle music often features a very irregular beat, rendering such tools inadequate. I am developing new methodologies for beat tracking of these complex rhythms, based not on machine learning, but on rules established from cognitive and musicological principles. Other aspects, such as distinguishing between the main melody and drone notes, need to be considered to obtain a complete music score. We are currently applying this technology to older recordings. The resulting music scores will be made available in the MEI format, which is becoming the new standard for written music representation. Additionally, this music score can be aligned with the audio recording of the performances, so the timing and other expressive aspects can be explicitly represented and studied further.

A longer-term objective is to design tools to dissect the musical corpus, aiming to extract key features of each tune. This initiative serves as an alternative to incipits and provides novel metadata formats, increasing usability connectivity within its content and with other databases. Another ambition is to develop advanced tools for browsing the music collection with interactive visualization and sonification. This would make the music and analyses accessible to the general public. One idea is to visualise the distribution of the entire catalogue based on intrinsic content, guided for instance by stylistic clustering, to help listeners of any expertise appreciate the richness of these catalogues.

In music analysis, the project with the National Library of Norway addresses issues of musical style, focusing on the intertextuality of music. One objective is to detect automatically the correspondences between pieces in the catalog (e.g. quotations, imitations, and other mimetic phenomena), to find patterns related to particular genre, dance, musician, region or period. For example, in the Norwegian folk music database, the project aims to detect melodies related to the same families or old religious songs. Advanced thematic catalogs could also be created, allowing visualization of the music collection and highlighting links between tunes. The idea is to enable an advanced music browser with hyperlinks that highlight possible links with other tunes and associated categories while visualizing and playing one tune. This would enable retrieval technology tailored to musicological queries, allowing musicologists to automatically find specific pieces of music based on their requests (e.g. a given theme or musical characteristic).

During the last twenty years of work on computational music analysis, I have been working on a comprehensive framework encompassing a wide range of music dimensions and various music representation formats, such as audio recordings, score and MIDI files; I have developed a well-known toolbox, MIRtoolbox,38 for extracting audio and music descriptors directly from audio. This experience has made me realize the limitations of analyzing audio recordings directly, underscoring the need to first transcribe audio into a score representation, and then perform more advanced musical analysis from that score. At the same time, those more advanced analyses help to enrich the transcription itself. Among the investigations are motivic and thematic analysis,39 ornamentation and reduction, phrasing detection, and modal and tonal analysis. The research leads to the development of a new software combining MIRtoolbox and score analysis: the MiningSuite.40

The Short-Term Scientific Mission (STSM) in Barcelona has fostered deep discussions on the pros and cons of different approaches in computational musicology. One key message I transmitted is that, while designing tools designed for specific use cases can be useful, general computational music analysis frameworks are also essential, particularly if they provide advanced analytical tools such as those developed in the Comprehensive Modelling Framework. Though specific music cultures, repertoires, or forms of music expression require particular analytical frameworks, my hypothesis is that these diverse musicological visions all rely on common cognitive mechanisms that can be adjusted through specific parameters. It must be acknowledged that this vision remains prospective, related to new developments in AI. The limited musicological usefulness of current AI tools and the skeptical assessment of AI potentials were discussed, enabling a balanced opinion concerning AI’s ideals and potentials in computational musicology. The technological advancement in text-based AI compared to music AI has been addressed, considering the contrasting difference in research efforts and datasets between those two domains. This can also be attributed to the particular difficulty of parsing music compared to text.

One initial objective of the STSM was to integrate layers of music analysis annotations into digital music edition projects, particularly Books of Hispanic Polyphony (BHP). One suggestion is to use the MEI music encoding format, leveraging existing music display technologies, and possibly tweaking them for music analysis display purposes. More importantly, this involves imagining the general principles of possible MEI customization for music analysis to foster the development of music analysis display from MEI visualizers.

The meeting in Barcelona has led to the establishment of ideas for collaboration in computational analysis of folk music, particularly towards: a) segmentation of field recordings of folk music of the Fondo de Música Tradicional IMF-CSIC (FMT) into songs; b) automated music transcriptions, encoding, and analysis of traditional music repertories (FMT and Ukrainian songs in particular); c) automated analysis of early music (BHP); and d) envisioning of technologies to visualize pieces and corpora, and to enable browsing based on music characteristics. Concerning Ukrainian songs, it is also considered the automated assessment of stanza pattern types. One concluding idea is to join European folk music collections into a common framework.

Anastasiia Mazurenko, Slovenian Academy of Science and Arts, Ljubljana