XML Unicode entities for non-Latin characters

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

XML Unicode entities for non-Latin characters

Paranoja
Hello Freeplane developers,

I noticed that Freeplane stores everything in ISO-8859-1 encoding. The difficulty arise when multiple users, who work on same mind-map file, stored in VCS, have to merge different versions by-hand. When non-latin characters are used in nodes, they are serialized as character entities and are difficult to read by human and makes merging such documents a hard task. Is there any hidden configuration parameter, which can be utilized to make Freeplane save mind-maps in UTF-8 encoding?

Regards,
Paranoja.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: XML Unicode entities for non-Latin characters

Paranoja
Sorry, my bad. After another thought about .mm files, it has been nowhere said that .mm file should be a valid xml.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: XML Unicode entities for non-Latin characters

Volker Börchers
Administrator
In reply to this post by Paranoja
Hi Paranoja,

.mm files are pure ASCII files. All non-ASCII characters are encoded
like this:

TEXT="BlaBla是一款手边娱乐利"
TEXT="lölich"

We had a discussion about changing the encoding but we didn't if I
remember correctly.

You actually have .mm files that contain non-ASCII characters?

Regards,
Volker

Am 12.02.2016 um 13:45 schrieb Paranoja [via Freeplane Developer]:

> Hello Freeplane developers,
>
> I noticed that Freeplane stores everything in ISO-8859-1 encoding. The
> difficulty arise when multiple users, who work on same mind-map file,
> stored in VCS, have to merge different versions by-hand. When non-latin
> characters are used in nodes, they are serialized as character entities
> and are difficult to read by human and makes merging such documents a
> hard task. Is there any hidden configuration parameter, which can be
> utilized to make Freeplane save mind-maps in UTF-8 encoding?
>
> Regards,
> Paranoja.
>
> ------------------------------------------------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://freeplane-developer.996965.n3.nabble.com/XML-Unicode-entities-for-non-Latin-characters-tp862.html
>
> To start a new topic under Freeplane Developer, email
> [hidden email]
> To unsubscribe from Freeplane Developer, click here
> <
> NAML
> <
http://freeplane-developer.996965.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: XML Unicode entities for non-Latin characters

Paranoja
No, I have all non-ASCII characters encoded in &#xabcd; encoding.
There is no bug with this, if you are asking.

The issue was with my understanding of file. We were putting .mm files
in Git repository. We were using non-ASCII characters in our
mind-maps. Because we were working on single file simultaneously,
sometimes we had to resolve merge conflicts and it was very hard to
understand what particular sequence of
&#abcd;&#abcd;&#abcd;&#abcd;&#abcd; means. So I thought it would be
good to ask wether Freeplane supports saving .mm files in UTF-8
encoding, as .mm file looks like valid XML file except there is no
<?xml ?> processing instruction at the beginning of the it.

Today I looked into a sources of Freeplane and now I know that there
is no such functionality in Freeplane. Also I tried to figure out how
difficult it would be to implement saving of .mm files in UTF-8
encoding. Looks like I found places where to change. If you are
interrested, I can implement that.

Regards,
Paranoja.


2016-02-12 23:10 GMT+02:00 Volker Börchers [via Freeplane Developer]
<[hidden email]>:

> Hi Paranoja,
>
> .mm files are pure ASCII files. All non-ASCII characters are encoded
> like this:
>
> TEXT="BlaBla&#x662f;&#x4e00;&#x6b3e;&#x624b;&#x8fb9;&#x5a31;&#x4e50;&#x5229;"
> TEXT="l&#xf6;lich"
>
> We had a discussion about changing the encoding but we didn't if I
> remember correctly.
>
> You actually have .mm files that contain non-ASCII characters?
>
> Regards,
> Volker
>
> Am 12.02.2016 um 13:45 schrieb Paranoja [via Freeplane Developer]:
>
>> Hello Freeplane developers,
>>
>> I noticed that Freeplane stores everything in ISO-8859-1 encoding. The
>> difficulty arise when multiple users, who work on same mind-map file,
>> stored in VCS, have to merge different versions by-hand. When non-latin
>> characters are used in nodes, they are serialized as character entities
>> and are difficult to read by human and makes merging such documents a
>> hard task. Is there any hidden configuration parameter, which can be
>> utilized to make Freeplane save mind-maps in UTF-8 encoding?
>>
>> Regards,
>> Paranoja.
>>
>> ------------------------------------------------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>>
>> http://freeplane-developer.996965.n3.nabble.com/XML-Unicode-entities-for-non-Latin-characters-tp862.html
>>
>> To start a new topic under Freeplane Developer, email
>> [hidden email]
>> To unsubscribe from Freeplane Developer, click here
>> <
>> NAML
>>
>> <http://freeplane-developer.996965.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://freeplane-developer.996965.n3.nabble.com/XML-Unicode-entities-for-non-Latin-characters-tp862p864.html
> To unsubscribe from XML Unicode entities for non-Latin characters, click
> here.
> NAML
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: XML Unicode entities for non-Latin characters

Dimitry Polivaev
Administrator
I think we have not changed it to stay compatible with FreeMind. But if you could give as an
implementation which recognizes <?xml ?> processing instruction and saves maps using in a given
encoding using an option (ascii means no instruction is necessary) we could include it into the source.

So feel free to fork us at the github and submit a pull request.

Regards,
Dimitry

> No, I have all non-ASCII characters encoded in &#xabcd; encoding.
> There is no bug with this, if you are asking.
>
> The issue was with my understanding of file. We were putting .mm files
> in Git repository. We were using non-ASCII characters in our
> mind-maps. Because we were working on single file simultaneously,
> sometimes we had to resolve merge conflicts and it was very hard to
> understand what particular sequence of
> &#abcd;&#abcd;&#abcd;&#abcd;&#abcd; means. So I thought it would be
> good to ask wether Freeplane supports saving .mm files in UTF-8
> encoding, as .mm file looks like valid XML file except there is no
> <?xml ?> processing instruction at the beginning of the it.
>
> Today I looked into a sources of Freeplane and now I know that there
> is no such functionality in Freeplane. Also I tried to figure out how
> difficult it would be to implement saving of .mm files in UTF-8
> encoding. Looks like I found places where to change. If you are
> interrested, I can implement that.
>
> Regards,
> Paranoja.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: XML Unicode entities for non-Latin characters

Felix Natter
Administrator
In reply to this post by Paranoja
"Paranoja [via Freeplane Developer]"
<[hidden email]> writes:

> No, I have all non-ASCII characters encoded in &#xabcd; encoding.
> There is no bug with this, if you are asking.

hello Paranoja,

> The issue was with my understanding of file. We were putting .mm files
> in Git repository. We were using non-ASCII characters in our
> mind-maps. Because we were working on single file simultaneously,
> sometimes we had to resolve merge conflicts and it was very hard to
> understand what particular sequence of
> &#abcd;&#abcd;&#abcd;&#abcd;&#abcd; means. So I thought it would be
> good to ask wether Freeplane supports saving .mm files in UTF-8
> encoding, as .mm file looks like valid XML file except there is no
> <?xml ?> processing instruction at the beginning of the it.
>
> Today I looked into a sources of Freeplane and now I know that there
> is no such functionality in Freeplane. Also I tried to figure out how
> difficult it would be to implement saving of .mm files in UTF-8
> encoding. Looks like I found places where to change. If you are
> interrested, I can implement that.

This author has implemented almost what you need, but for freemind (but
I think at this level FM and FP are pretty similar):
  https://github.com/jiangxin/freemind-mmx

I would welcome such a change if it's optional so that the existing save
still works and is not changed/broken.

I think this, based on Dimitry's idea, is good:
- UTF-8 save for new files is configurable (prefs), but
- if you load a map saved with UTF-8 (i.e.  <?xml version="1.0"
  encoding="UTF-8"?>), then it will always save as UTF-8

What do the devs think?

Happy Easter and Best Regards,
--
Felix Natter
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: XML Unicode entities for non-Latin characters

Paranoja
Hello Felix,

I was looking into Freeplane master and tried to figure out how to
properly add UTF-8 support, but was distracted away. Thanks for your
reference to freemind-mmx. I'll look into it to see how UTF-8 was
implemented there.


2016-03-26 17:14 GMT+02:00 Felix Natter [via Freeplane Developer]
<[hidden email]>:

> "Paranoja [via Freeplane Developer]"
> <[hidden email]> writes:
>
>> No, I have all non-ASCII characters encoded in &#xabcd; encoding.
>> There is no bug with this, if you are asking.
>
> hello Paranoja,
>
>> The issue was with my understanding of file. We were putting .mm files
>> in Git repository. We were using non-ASCII characters in our
>> mind-maps. Because we were working on single file simultaneously,
>> sometimes we had to resolve merge conflicts and it was very hard to
>> understand what particular sequence of
>> &#abcd;&#abcd;&#abcd;&#abcd;&#abcd; means. So I thought it would be
>> good to ask wether Freeplane supports saving .mm files in UTF-8
>> encoding, as .mm file looks like valid XML file except there is no
>> <?xml ?> processing instruction at the beginning of the it.
>>
>> Today I looked into a sources of Freeplane and now I know that there
>> is no such functionality in Freeplane. Also I tried to figure out how
>> difficult it would be to implement saving of .mm files in UTF-8
>> encoding. Looks like I found places where to change. If you are
>> interrested, I can implement that.
>
> This author has implemented almost what you need, but for freemind (but
> I think at this level FM and FP are pretty similar):
>   https://github.com/jiangxin/freemind-mmx
>
> I would welcome such a change if it's optional so that the existing save
> still works and is not changed/broken.
>
> I think this, based on Dimitry's idea, is good:
> - UTF-8 save for new files is configurable (prefs), but
> - if you load a map saved with UTF-8 (i.e.  <?xml version="1.0"
>   encoding="UTF-8"?>), then it will always save as UTF-8
>
> What do the devs think?
>
> Happy Easter and Best Regards,
> --
> Felix Natter
>
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://freeplane-developer.996965.n3.nabble.com/XML-Unicode-entities-for-non-Latin-characters-tp862p901.html
> To unsubscribe from XML Unicode entities for non-Latin characters, click
> here.
> NAML
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: XML Unicode entities for non-Latin characters

Felix Natter
Administrator
"Paranoja [via Freeplane Developer]"
<[hidden email]> writes:

> Hello Felix,

hello Paranoja,

> I was looking into Freeplane master and tried to figure out how to
> properly add UTF-8 support, but was distracted away. Thanks for your
> reference to freemind-mmx. I'll look into it to see how UTF-8 was
> implemented there.

Please use branch '1.5.x', which is the HEAD branch, 'master'
is outdated.

Cheers and Best Regards,
--
Felix Natter
Loading...