Page MenuHomeSoftware Heritage

Review IPOL deposit metadata
Closed, ResolvedPublic

Description

IPOL editor has deposited the first IPOL deposit.

We need to make sure that the artifacts created on SWH and the associated metadata are correct (corresponds to the metadata on the website).

The origin
This is the origin: https://www.ipol.im/87afca27-ebbb-408a-845d-5277b6a92f49
Which is not a working URL -> error 404
There is a DOI: https://doi.org/10.5201/ipol.2018.236
which can be the origin ?!?
Or this url: https://www.ipol.im/pub/art/2018/236/

The snapshot
https://archive.softwareheritage.org/browse/origin/https://www.ipol.im/87afca27-ebbb-408a-845d-5277b6a92f49/branches/
Branch HEAD is created

The revision
Here is the object on SWH:
swh:1:rev:86e6c7e3ad0699601f73f343344a65f65b6f76ce;origin=https://www.ipol.im/87afca27-ebbb-408a-845d-5277b6a92f49
this is a synthetic revision
author and committer are: `Software Heritage'
with the same author_date and commiter_date resolved with T2368

metadata:

}
    "@xmlns": "http://www.w3.org/2005/Atom",
    "@xmlns:codemeta": "https://doi.org/10.5063/SCHEMA/CODEMETA-2.0",
    "author": "Jose-Luis Lisani",
    "codemeta:applicationCategory": "Image Processing On Line (IPOL)",
    "codemeta:dateCreated": "2018-11-14",
    "codemeta:datePublished": "2018-12-07",
    "codemeta:description": "Implementation of Shape Preserving Local Histogram Modification Algorithm",
    "codemeta:identifier": "https://doi.org/10.5201/ipol.2018.236",
    "codemeta:keywords": [
        "contrast enhancement",
        "histogram modification",
        "local processing"
    ],
    "codemeta:license": {
        "codemeta:name": "LGPL-3.0-or-later",
        "codemeta:url": "https://spdx.org/licenses/LGPL-3.0-or-later.html"
    },
    "codemeta:operatingSystem": "Linux",
    "codemeta:programmingLanguage": "C++",
    "codemeta:referencePublication": {
        "codemeta:identifier": "236",
        "codemeta:name": "An Analysis and Implementation of the Shape Preserving\nLocal Histogram Modification Algorithm"
    },
    "codemeta:releaseNotes": "On-line demo available at IPOL webpage",
    "codemeta:url": "https://www.ipol.im/pub/art/2018/236/",
    "external_identifier": "236",
    "extrinsic": {
        "provider": "https://deposit.softwareheritage.org/1/private/551/meta/",
        "raw": {
            "branch_name": "master",
            "origin": {
                "type": "deposit",
                "url": "https://www.ipol.im/87afca27-ebbb-408a-845d-5277b6a92f49"
            },
            "origin_metadata": {
                "metadata": {
                    "@xmlns": "http://www.w3.org/2005/Atom",
                    "@xmlns:codemeta": "https://doi.org/10.5063/SCHEMA/CODEMETA-2.0",
                    "author": "Jose-Luis Lisani",
                    "codemeta:applicationCategory": "Image Processing On Line (IPOL)",
                    "codemeta:dateCreated": "2018-11-14",
                    "codemeta:datePublished": "2018-12-07",
                    "codemeta:description": "Implementation of Shape Preserving Local Histogram Modification Algorithm",
                    "codemeta:identifier": "https://doi.org/10.5201/ipol.2018.236",
                    "codemeta:keywords": [
                        "contrast enhancement",
                        "histogram modification",
                        "local processing"
                    ],
                    "codemeta:license": {
                        "codemeta:name": "LGPL-3.0-or-later",
                        "codemeta:url": "https://spdx.org/licenses/LGPL-3.0-or-later.html"
                    },
                    "codemeta:operatingSystem": "Linux",
                    "codemeta:programmingLanguage": "C++",
                    "codemeta:referencePublication": {
                        "codemeta:identifier": "236",
                        "codemeta:name": "An Analysis and Implementation of the Shape Preserving\nLocal Histogram Modification Algorithm"
                    },
                    "codemeta:releaseNotes": "On-line demo available at IPOL webpage",
                    "codemeta:url": "https://www.ipol.im/pub/art/2018/236/",
                    "external_identifier": "236",
                    "title": "mlheIPOL"
                },
                "provider": {
                    "metadata": {},
                    "provider_name": "",
                    "provider_type": "deposit_client",
                    "provider_url": "https://www.ipol.im/"
                },
                "tool": {
                    "configuration": {
                        "sword_version": 2
                    },
                    "name": "swh-deposit",
                    "version": "0.0.1"
                }
            }
        },
        "when": "2020-04-20T23:16:11.436881+00:00"
    },
    "original_artifact": [
        {
            "checksums": {
                "sha1": "cdce42697925513b15824f69bd461ec6f3bc9571",
                "sha256": "2a261e6b673f09f60185f6fbf874a828d3b5aeee946d38dd2f2bc4d899db0b1a"
            },
            "filename": "archive.zip",
            "length": 661595
        }
    ],
    "title": "mlheIPOL"
}

The release
Not yet implemented

The directory
[TODO]

Event Timeline

moranegg updated the task description. (Show Details)

The extrinsic property seems like a bug. needs further analysis.

moranegg changed the task status from Open to Work in Progress.Apr 21 2020, 12:34 PM
moranegg claimed this task.
moranegg triaged this task as High priority.


metadata file

From the article web page:

published
    2018-12-07 
reference
    Jose-Luis Lisani, An Analysis and Implementation of the Shape Preserving Local Histogram Modification Algorithm, Image Processing On Line, 8 (2018), pp. 408–434. https://doi.org/10.5201/ipol.2018.236

Communicated by Jean-Michel Morel, Miguel Colom
Demo edited by Jose-Luis Lisani
Abstract

In this paper we describe the implementation of the algorithm for local contrast enhancement published by Caselles et al. in 1999. This algorithm was the first designed explicitly to increase the contrast while preserving the so-called 'shape structure' of the image, that is, its set of level sets. According to the mathematical morphology school, artifacts are created when this structure is modified. The original algorithm is described and also two alternative implementations are proposed, which limit the over-enhancement of noise.
Download

    full text manuscript: PDF low-res. (957KB) PDF (92.7MB) [?]
    source code: TAR/GZ

from the demo web page:

Description

Given an input image this demo computes the result of the three versions of the MLHE algorithm described in the associated article: MLHE HE, MLHE PAE and MLHE CLAHE. The users can select the parameters of each method. The default parameters are the same described in the article. For comparison, the results of classical histogram equalization (HE) and CLAHE are also displayed. We use obtained the Matlab implementation of the CLAHE algorithm. Both HE and CLAHE are applied on the intensity component of the input image and the output color image is obtained as in the MLHE methods.
Select input(s)
Input(s)
Parameters

From BibTeX:

@article{ipol.2018.236,
    title   = {{An Analysis and Implementation of the Shape Preserving Local Histogram 
Modification Algorithm}},
    author  = {Lisani, Jose-Luis},
    journal = {{Image Processing On Line}},
    volume  = {8},
    pages   = {408--434},
    year    = {2018},
    doi     = {10.5201/ipol.2018.236},
}
% if your bibliography style doesn't support doi fields:
    note    = {\url{https://doi.org/10.5201/ipol.2018.236}}
moranegg updated the task description. (Show Details)Apr 21 2020, 6:40 PM

The origin
For IPOL, we could use indifferently the DOI: https://doi.org/10.5201/ipol.2018.236 or the URL: https://www.ipol.im/pub/art/2018/236/, as it is trivial to convert the one into the other.

More generally, though, a DOI may be more resilient on the long term, and the wayback machine seems also now to resolve DOIs, so let's go for DOIs as the origin (it's the "paper" after all!)

zack added a subscriber: zack.Apr 21 2020, 8:51 PM

If we go for the DOI, shouldn't we converge on it also for other similarly archives, for the sake of uniformity? HAL comes to mind, of course.
(To be clear: I'm not proposing to uniform past deposits, but to change this for the future.)

In T2369#43616, @zack wrote:

If we go for the DOI, shouldn't we converge on it also for other similarly archives, for the sake of uniformity? HAL comes to mind, of course.
(To be clear: I'm not proposing to uniform past deposits, but to change this for the future.)

HAL does not give DOIs, but HAL_IDs, which are exactly what we are using as origin right now, for the software deposits.

It is too early to see any general pattern here, let's wait until we have a few more concrete examples.

One difference between HAL and IPOL is the item identified with the identifier:

  • In HAL, the HAL-ID is given to the software artifact alone and if you want to deposit the main software article it will have a another identifier.
  • In IPOL, the DOI identifies the metadata record of both the published article and the published code

Also the IPOL article and code are peer reviewed which results with a publication with an IPOL DOI.

The choice for IPOL is between:

It makes sense to have this choice also for the origin of a Zenodo deposit.

The choice for IPOL is between:
[1] https://www.ipol.im/pub/art/2018/236/ -> link to metadata record containing article, code and demo
[2] https://www.ipol.im/pub/art/2018/236/mlheIPOL.tgz -> direct link to tar.gz
[3] https://doi.org/10.5201/ipol.2018.236 -> which is a PID recognized in the academic ecosystem

We want of course to point to the web page that contains the metadata and the link to the code (exactly like we do for HAL or GitHub, etc.), so [2] is not an option.
We are left with [1] and [3], which are, as said before, quite trivial to interconvert in this specific case (it is not true in general!)
As said before, DOIs may be more resilient on the long term, and the wayback machine seems also now to resolve DOIs (I checked this particular case).

So let's go for [3] and suggest to the IPOL team use DOIs as the origin for their deposits.

For other Journals and Publishers, and even Zenodo: if a DOI is available, we'll recommend it, but some do not provide DOIs, and we'll have to look case by case.

moranegg updated the task description. (Show Details)Apr 23 2020, 4:40 PM
moranegg added a comment.EditedApr 23 2020, 5:43 PM

I suggest the following changes:

  1. change applicationCategory = Image processing
  2. add isPartOf = Image Processing On Line (IPOL) or isPartOf = IPOL
  3. add downloadUrl = https://www.ipol.im/pub/art/2018/236/mlheIPOL.tgz
  4. add DOI and abstract in referencePublication :
<codemeta:referencePublication>
          <codemeta:name>An Analysis and Implementation of the Shape Preserving
Local Histogram Modification Algorithm</codemeta:name>
          <codemeta:identifier>https://doi.org/10.5201/ipol.2018.236</codemeta:identifier>
          <codemeta:url>https://www.ipol.im/pub/art/2018/236/article_lr.pdf </codemeta:url>
          <codemeta:abstract> a place for the article abstract </codemeta:abstract>
        </codemeta:referencePublication>
  1. add the demo url as relatedLink= http://ipolcore.ipol.im/demo/clientApp/demo.html?id=236
  2. rewrite releaseNotes- should be about the release, but I still need to think about that, because with that metadata property, we aim to create releases on the SWH archive.
  3. change external_identifier to ipol.2018.236 (which is actually the way we recognize that a deposit made by the same client with the same external_identifier, is a new version, it might be more useful to have 2018-236 instead of just 236.

and sent the following questions:

  1. How do you choose the title for the software?
  2. Also where will you find the description, would it be the demo description?
  3. There are three identified actors on the IPOL webpage:
    • Communicated by
    • Demo edited
    • Author (in the BibTeX and reference)

Only the author is in the xml, should we find corresponding properties for the other two actors?

  1. I'm very happy about the keywords, license, programmingLanguage and operatingSystem properties. With that, I didn't find this metadata on the IPOL webpage, could you have these properties on all deposits?

Here is the new IPOL proposition (made by José-Luis):

<?xml version="1.0"?>
        <entry xmlns="http://www.w3.org/2005/Atom"
               xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0">
        <title>mlheIPOL</title>
        <external_identifier>2018-236</external_identifier>
        <author>Jose-Luis Lisani</author>

        <codemeta:url>https://www.ipol.im/pub/art/2018/236/</codemeta:url>
        <codemeta:identifier>https://doi.org/10.5201/ipol.2018.236</codemeta:identifier>
        <codemeta:applicationCategory>Image Processing On Line (IPOL)</codemeta:applicationCategory>

        <codemeta:description>Implementation of Shape Preserving Local Histogram Modification Algorithm</codemeta:description>
        <codemeta:keywords>contrast enhancement</codemeta:keywords>
        <codemeta:keywords>histogram modification</codemeta:keywords>
        <codemeta:keywords>local processing</codemeta:keywords>
        <codemeta:dateCreated>2018-11-14</codemeta:dateCreated>
        <codemeta:datePublished>2018-12-07</codemeta:datePublished>
        <codemeta:releaseNotes>On-line demo available at IPOL webpage</codemeta:releaseNotes>
        <codemeta:referencePublication>
            <codemeta:name>An Analysis and Implementation of the Shape Preserving
                Local Histogram Modification Algorithm</codemeta:name>
            <codemeta:identifier>https://doi.org/10.5201/ipol.2018.236</codemeta:identifier>
            <codemeta:url>https://www.ipol.im/pub/art/2018/236/article_lr.pdf </codemeta:url>
            <codemeta:abstract>In this paper we describe the implementation of the algorithm for local contrast enhancement published by Caselles et al. in 1999. This algorithm was the first designed explicitly to increase the contrast while preserving the so-called 'shape structure' of the image, that is, its set of level sets. According to the mathematical morphology school, artifacts are created when this structure is modified. The original algorithm is described and also two alternative implementations are proposed, which limit the over-enhancement of noise.</codemeta:abstract>
        </codemeta:referencePublication>
        <codemeta:isPartOf>
            <codemeta:type>Journal</codemeta:type>
            <codemeta:name>Image Processing On Line (IPOL)</codemeta:name>
            <codemeta:identifier> ISSN: 2105-1232 DOI: 10.5201/ipol </codemeta:identifier>
        </codemeta:isPartOf>
        <codemeta:relatedLink>http://ipolcore.ipol.im/demo/clientApp/demo.html?id=236</codemeta:relatedLink>

        <codemeta:license>
            <codemeta:name>LGPL-3.0-or-later</codemeta:name>
            <codemeta:url>https://spdx.org/licenses/LGPL-3.0-or-later.html</codemeta:url>
        </codemeta:license>

       <codemeta:programmingLanguage>C++</codemeta:programmingLanguage>

        <codemeta:operatingSystem>Linux</codemeta:operatingSystem>

</entry>
moranegg added a subscriber: ardumont.EditedApr 27 2020, 9:53 PM

@ardumont can we have dots in a deposit slug / external_identifier?
An example:
ipol.2018.236

moranegg added a comment.EditedApr 27 2020, 9:56 PM

Still needs review:

@ardumont can we have dots in a deposit slug / external_identifier?
An example:
ipol.2018.236

yes. It's a text field.

> \d deposit
        Column         |           Type           | Collation | Nullable |               Default
-----------------------+--------------------------+-----------+----------+-------------------------------------
...
 external_id           | text                     |           | not null |
...

Still needs review:

  • external_identifier

use ipol.<year>.<id> like used in the DOI identifier

  • have the author in a CodeMeta field with affiliation?
<codemeta:author>
  <codemeta:name> name <\codemeta:name>
  <codemeta:affiliation> affiliation<\codemeta:affiliation>
<\codemeta:author>
  • releaseNotes

use demo description

  • url (for origin)

use https://doi.org/10.5201/ipol.2018.236
with https://doi.org/10.5201/ as provider_url
and ipol.2018.236 as Slug or external_id

url the complete url https://doi.org/10.5201/ipol.2018.236
domain doi.org/10.5201/

yes

  • is there a notion of version

There is no notion of version, but we should ask José-Luis about that.

Instructions given to José-Luis with the following files:

With the new deposits on May 7th we have:

origin

The origin: https://doi.org/10.5201/ipol.2018.236
https://archive.softwareheritage.org/browse/origin/visits/?origin_url=https://doi.org/10.5201/ipol.2018.236

snapshots/ visits

3 visits at 00.53 = 3 snapshots
all are linked to the same origin, but the revisions do not appear in history

branch

  • 1 branch HEAD for 2 deposits
  • 2 branches for swh:1:snp:206991563b219d18cdfc998d0e35716ba142c12e;origin=https://doi.org/10.5201/ipol.2018.236
    • one HEAD
    • one with the revision SWHID

revision

3 different revision with in the metadata:
version = 1.0
version =1.1
version = 1.2
In the readme:
Version 3.0 - November 13, 2018
same directory target for the three versions: a10423592dd061a00f7d34e4a3c102ba00c3d2ab

Metadata

See https://forge.softwareheritage.org/P665
Remark:
"codemeta:applicationCategory": "Image Processing On Line (IPOL)"
should become
"codemeta:applicationCategory": "Image Processing"

all are linked to the same origin, but the revisions do not appear in history

11:49 <+ardumont> i think it's more that the 3 deposits were done very closely
11:49 <+ardumont> and that the chaining happens if you use the same slug and that you have a deposit done already
11:49 <+ardumont> which with the current slugishness scheduler side cannot happen
11:49 <+ardumont> moranegg[m]: i'm more and more convinced that the issue is that the 3 deposits (which are only minute-separated) where checked and injected almost at the same time
11:52 <+ardumont> tl; dr; for the chaining, slug is a necessary condition but it's not sufficient ¯\_(ツ)_/¯

injected almost at the same time

I think the following qualifies... :D

softwareheritage=> select ov.* from origin_visit ov inner join origin o on o.id=ov.origin where ov.type='deposit' and o.url='https://doi.org/10.5201/ipol.2018.236';
  origin   | visit |             date              | status | metadata |                  snapshot                  |  type
-----------+-------+-------------------------------+--------+----------+--------------------------------------------+---------
 119719296 |     3 | 2020-05-07 00:53:02.036759+00 | full   |          | \x206991563b219d18cdfc998d0e35716ba142c12e | deposit
 119719296 |     2 | 2020-05-07 00:53:01.770335+00 | full   |          | \x3ceae6836c8fa31e09d66133934dbc2c68ac7860 | deposit
 119719296 |     1 | 2020-05-07 00:53:01.636065+00 | full   |          | \xf7decde6a26a4fa5f0886d71c010ceae827bae92 | deposit
(3 rows)

@ardumont , @moranegg : may you check that everything is now ok? We should close this task it that's the case :-)

Well, from my point of view, it is (only answering to her last remaining question).
I will let @moranegg concludes though (as she opened it in the first place).

moranegg closed this task as Resolved.May 14 2020, 5:14 PM

P668 and P672 show that metadata is ok now.
I'm still a bit worried about linking between parent and child revisions with the same --slug (which do not appear on the history page in the WEB-UI).
@ardumont can you verify if how the revision below looks like on the deposit DB?
https://archive.softwareheritage.org/swh:1:rev:e8c3b521316edb2840078fae44cad9a66d5f5fbb;origin=https://doi.org/10.5201/ipol.2018.236;visit=swh:1:snp:07c80b96ab64e714fb69ed725f6b18caf87763ba/