View Issue Details

IDProjectCategoryView StatusLast Update
0002926Ham Radio DeluxeMaintenancepublic2020-08-03 03:14
ReporterPD9FERAssigned ToDOUG 
PrioritynormalSeveritytweakReproducibilityalways
Status closedResolutionreopened 
Product Version 
Target VersionFixed in Version6.7.0.301 
Summary0002926: New and altered IOTA data in Logbook
Descriptionhttps://www.iota-world.org/info/new_groups_2018.pdf

There are some changes/additions in the IOTA designators (see link above)
Customer worked a station in IOTA NA-249P but it is not in the list.
Steps To ReproduceOpen ALE
Open the IOTA List dropdown
Look for NA-249 It is not there.

Additional InformationTicket #874915

Changes where made, use the link https://www.iota-world.org/info/new_groups_2018.pdf
for those.

The file was provided by the IOTA site on RSGB in XML format (filename: fulllist.xml). Periodically, we downloaded it and made the update. It seems that there are new entries to this list.

TagsNo tags attached.
ModuleLogbook
Sub-ModuleIOTA
Testing Beta Failed

Activities

WA9PIE

2019-02-15 11:02

administrator   ~0007401

We've had a few more tickets on this topic and we should address it.

KB3NPH

2019-03-23 07:26

administrator   ~0007724

Ticket #827216 just received about when and how the IOTA in HRD would be updated.

K7ZCZ

2019-04-09 11:18

developer   ~0007845

Where can I find documentation on the suggested API?

K7ZCZ

2019-04-09 12:05

developer   ~0007846

I've dug around on their site a bit, and this is all I can find:
https://www.iota-world.org/management-news/896-calling-software-developers.html

so I've written the email address given there to request information about their API, licensing, and acceptable use.

K7ZCZ

2019-04-11 05:18

developer   ~0007847

I got in touch with the IOTA team. A given API key is good for only 1000 queries per day, so we'll need to build a reflector -- in the same way we have for the solar weather data. We'll need to decide how that works, how often it updates, and how it is monitored.

Architecturally, we'll need to figure out how to handle the format change; the existing code expects an XML format for IOTA data, while the new format is JSON. Does the forwarding service provide XML, or JSON, or a completely new format? It's not much fun parsing JSON in C++, so it seems like we should stick with XML or invent our own format -- seems like CSV would be fine, and simpler and faster than XML. Either way, if we're not sending JSON to the applications, then we need to have code translate the JSON into the redistributed format rather than simply serve a specific file.

I'm not sure how these decisions should/would be addressed by the team.

WA9PIE

2019-05-10 18:31

administrator   ~0007929

Doesn't this API essentially build the list? And if so, can't we build the list once... at the time we compile code?

K7ZCZ

2019-05-12 20:52

developer   ~0007931

If we build the list when the code is complied, then we won't pick up changes until the next version is compiled. Customers won't pick up the changes until the next time they install a new version. Building new versions of code to accommodate changes in external data doesn't seem like a good pattern to me.

Plus, as I detail above, the API produces data in a format that we're not prepared to consume.

K7ZCZ

2019-05-13 12:57

developer   ~0007932

Attached are the IOTA docs I was sent. They're very scant--in particular, no meaningful definitions for each field in the data that's provided by the API.

On our side, we don't have any documentation for the existing IOTA XML format, or for any of the IOTA features themselves. This means we have to go through the code to figure out what the data does and what it's used for. Then, we'll need to get data from the new IOTA API and compare it to what we can infer about the old XML format to see if there's a direct replacement or not ... or if some translation is needed.

K7ZCZ

2019-05-13 12:58

developer   ~0007933

second try at the attachment ...

K7ZCZ

2019-05-13 12:59

developer   ~0007934

Looks like attachments aren't working right for me :(

PD9FER

2019-11-29 04:27

viewer   ~0009479

Ticket # 244581

Customer comment:
If maintaining the IOTA list is considered an enhancement it would be nice if I could manually enter this information. Otherwise I can't maintain an accurate log.

Also, here are more IOTA's that are missing from the list.
EU-192 OJ9, SM Kataja/Inakari Island

NA-249 KP3, 4 Puerto Rico's Coastal Islands
NA-250 KL Yakutat Borough group

OC-297 FO Morane Atoll
OC-298 FO Tatakoto Atoll
OC-299 V6 Yap East Group
OC-300 T31 McKean and Nikumaroro Atolls

SA-101 CE0 Juan Fernandez Archip (Alejandro Selkirk)

WA9PIE

2019-12-02 00:24

administrator  

response.json (1,356,459 bytes)

WA9PIE

2019-12-02 00:24

administrator   ~0009484

Going back to the post I made regarding the API key, I'm attaching the resulting json file.

We need to add this to the build script so that this is pulled and updated whenever we do a build. We just need to figure out how to ingest the json file as attached.

This replaces a file in resource called something like iota.xml.

Mike

WA9PIE

2019-12-05 17:49

administrator   ~0009488

Last edited: 2019-12-05 17:50

View 2 revisions

Basically, one of these things is what needs to happen:

Option 1:
- at the time of build, we hit this URL with this API key
- obtain the json file
- reformat it into the format currently in Logbook - OR modify Logbook to consume the json file directly
- add it into the build as a resource
- done

Option 2:
- we modify Logbook so that it can consume the json file directly
- we let Logbook update the IOTA file in a similar way as how the Country List gets updated now
- done
(this gets the updates to the users more quickly without the need for a new build; the downside is that it could expose our API key)

I'm not sure I have a preference. But option 2 has an added benefit.

Thoughts?

KB3NPH

2020-02-04 05:58

administrator   ~0009579

Ticket #672294 also indicates NA-249 missing from our current IOTA List.

PD9FER

2020-04-08 05:45

viewer   ~0009662

New Ticket #669380

DOUG

2020-07-11 22:06

developer   ~0009742

fiixed in version 289

This version is has the json file built into the install. It would be pretty easy to update it from the api call, but I haven't added that yet.

vk2byi

2020-07-13 03:46

viewer   ~0009758

All Islands as per IOTA Addendum (link in Description) correctly appear in the listbox as per their corresponding IOTA Group.

NOTE: There is a Ryponkicha listbox entry under the AS-204 Group, yet that name is **NOT** listed as part of this group in the IOTA Directory:

    "AS-204 R0F c. KURIL'SKIYE ISLANDS (KURIL ISLANDS) NORTH
            (=Antsiferova, Atlasov, Chirinkotan, Ekarma, Ketoy, Kharimkotan, Makanrushi, Matua,
                      Onekotan, Paramushir, Raikoke, Rasshua, Shiashkotan, Shumshu, Simushir, **Ushishir** )"

However, Google and Wikipedia tell me that Ryponkicha and Yankicha are two islets that make up **Ushishir** island. As Ryponkicha is the major islet of the two, this seems reasonable. Yet Yankicha is not in the listbox, and despite what the website IOTA Directory shows, maybe this is how it is structured in the response.json built into this install. Certainly not an an issue, just a heads up.

I call this a testing pass for this issue.

vk2byi

2020-07-13 03:48

viewer   ~0009759

All Islands as per IOTA Addendum (link in Description) correctly appear in the listbox as per their corresponding IOTA Group.

NOTE: There is a Ryponkicha listbox entry under the AS-204 Group, yet that name is **NOT** listed as part of this group in the IOTA Directory:

    "AS-204 R0F c. KURIL'SKIYE ISLANDS (KURIL ISLANDS) NORTH
                  (=Antsiferova, Atlasov, Chirinkotan, Ekarma, Ketoy, Kharimkotan, Makanrushi, Matua,
                   Onekotan, Paramushir, Raikoke, Rasshua, Shiashkotan, Shumshu, Simushir, **Ushishir** )"

However, Google and Wikipedia tell me that Ryponkicha and Yankicha are two islets that make up **Ushishir** island. As Ryponkicha is the major islet of the two, this seems reasonable. Yet Yankicha is not in the listbox, and despite what the website IOTA Directory shows, maybe this is how it is structured in the response.json built into this install. Certainly not an an issue, just a heads up.

I call this a testing pass for this issue.

vk2byi

2020-07-13 03:51

viewer   ~0009762

I got an error posting my test note above, tried again and received the same 'Slack curl' error but ended up with duplicate posts. I can't delete the duplicate entry.

vk2byi

2020-07-13 03:52

viewer   ~0009764

I got an error posting my test note above, tried again and received the same 'Slack curl' error but ended up with duplicate posts. I can't delete the duplicate entry.

g3ucq

2020-07-13 03:57

viewer   ~0009766

Fixed for me.
Also getting Slack errors

WA9PIE

2020-07-13 05:06

administrator   ~0009781

Yep, this change is validated.

I tested this by looking at the most recent IOTA island announcement I could find: https://www.dx-world.net/new-iota-groups-announced-2019/

Also fixed the Slack posting errors.

WA9PIE

2020-07-13 05:30

administrator   ~0009785

Chris had an update to make about this. So I'm bouncing this back until we review it.

vk2byi

2020-07-13 19:49

viewer   ~0009792

After a quick browse of the Management News on the IOTA website (https://www.iota-world.org/management-news.html), I believe IOTA changes since 14 Oct 2018 have occurred in Groups AS-204, AS-205, EU-192, NA-249, NA-250 and OC-297, as well as AS-177 and AS-206.

AS-204, AS-205, EU-192, NA-249, NA-250 and OC-297 all have the correct list of islands displayed in the Island listbox. AS-204 also includes Ryponkicha as mentioned previously, but that shouldn't matter.

Also, during testing I noticed that there are no entries in the Island listbox for Group AS-207 which should contain Kosa, Dvukh, and Pilotov.

Outstanding issues are then as follows:
    AS-117: contains outdated list of islands, including those that should move to AS-206
    AS-206: no entries in Island listbox
    AS-207: no entries in Island listbox

WA9PIE

2020-07-13 20:05

administrator   ~0009793

Doug had provided me with a "side-by-side" copy of the old data and the new data. Let's take a look at this with regards to the points made by @vk2byi

iota2-json.xlsx (195,973 bytes)

vk2byi

2020-07-13 20:13

viewer   ~0009794

Restructuring of AS-117/AS-206 is mentioned by IOTA here: https://www.iota-world.org/management-news/937-restructuring-of-as-117-as-206.html

WA9PIE

2020-07-13 20:19

administrator   ~0009795

Yep... what's interesting here is this...

The folks who run IOTA now went from an old file structure to a JSON file that (as far as I know) gets generated with some sort of JSON call to a system managed by IOTA World. So the file we're looking at here was generated by this JSON call. (again, not being a developer, I don't know how this works in detail) That said... you'd think, if this file is generated from a JSON call directly to their system... that there wouldn't be any problems with the content?

WA9PIE

2020-07-21 06:26

administrator   ~0009813

Okay...

The changes that @vk2byi is referring to here, referenced in the restructuring announcement, are not reflected in the JSON file.

Things to ponder:
1 - If this is the JSON data that was obtained directly from the source, and it does not contain the correct contents for AS206 and AS-207, then... can/should WE fix this? Or should we report this to the IOTA org and see if they'll fix it later? (In other words - if our code that goes off to obtain the JSON file works, should we spend time trying to fix the errors in the data obtained from the source?)
2 - if this is not the JSON data that was obtained directly from the source, then... are we doing something with this data to cause it to not reflect the correct data?

If #1, then we should probably ship this and let the IOTA folks sort out their data.
If #2, then we should fix this before we ship.

Thoughts? (I have no way to look at the original data. It requires a key and code to get directly to the JSON file.)

IOTAchanges.png (285,336 bytes)

g3ucq

2020-07-21 09:17

viewer   ~0009816

Either 1 or 2, is it possible to update in Logbook QSOs that took place BEFORE the IOTA modifications?
If not, the #1.

WA9PIE

2020-07-28 03:33

administrator   ~0009825

I checked this one in the 6.7.0.296 build and the problem still exists (regarding the contents of AS-117, AS-206, & AS-207).

Will hold the release for resolving this.

WA9PIE

2020-07-31 11:54

administrator   ~0009837

The detail for these three island groups can be found here:

AS-117; https://www.iota-world.org/islands-on-the-air/iota-groups-islands/group/248.html
AS-206; https://www.iota-world.org/islands-on-the-air/iota-groups-islands/group/1179.html
AS-207; https://www.iota-world.org/islands-on-the-air/iota-groups-islands/group/1178.html

If the file that comes from the IOTA source is incomplete, I can probably move the data around so we have it correct per these references.

g3ucq

2020-07-31 14:31

viewer   ~0009838

I have worked plenty of IOTA stations 'not on the list' but should a release containing FTDX101 held up for a solution to this problem?

WA9PIE

2020-08-01 00:13

administrator   ~0009839

The hold on the release is 95% related to waiting for Digicert to renew our code signing certificate. This should come early next week.

WA9PIE

2020-08-01 00:18

administrator   ~0009840

The specific changes that need to be made to the IOTA list at this point (difference to the JSON I saw last) are found in the attached file.

IOTA Specific Changes.txt (4,050 bytes)
AS-117

Comments
Not Mukai [in Hiroshima]; Hiko [in Yamaguchi]; Omi [in Yamaguchi] or islands in the Seto Naikai which fall in the JA5 call area [AS-076 and AS-200]

Islands
Abuga (JA9 Toyama Prefecture),Ai (JA4 Yamaguchi Prefecture),Ao (JA4 Okayama Prefecture),Atata (JA4 Hiroshima Prefecture),Awaji (JA3 Hyogo Prefecture),Benten (JA4 Hiroshima Prefecture),Boze (JA3 Hyogo Prefecture),Eta (JA4 Hiroshima Prefecture),Fune (JA4 Hiroshima Prefecture),Ganryu (JA4 Yamaguchi Prefecture),Ha (JA4 Yamaguchi Prefecture),Hane (JA4 Yamaguchi Prefecture),Hashira (JA4 Yamaguchi Prefecture),Hashiri (JA4 Hiroshima Prefecture),Hegura (JA9 Ishikawa Prefecture),Heigun (JA4 Yamaguchi Prefecture),Hodaka (JA4 Yamaguchi Prefecture),I (JA4 Okayama Prefecture),Ie (JA3 Hyogo Prefecture),Ikuchi (JA4 Hiroshima Prefecture),Inno (JA4 Hiroshima Prefecture),Inu (JA4 Okayama Prefecture),Iseko (JA4 Yamaguchi Prefecture),Iwai (JA4 Yamaguchi Prefecture),Jino (JA3 Wakayama Prefecture),Ka (JA4 Hiroshima Prefecture),Kakui (JA4 Okayama Prefecture),Kamiguro (JA4 Hiroshima Prefecture),Kamikamagari (JA4 Hiroshima Prefecture),Kamimizu (JA4 Okayama Prefecture),Kanawa (JA4 Hiroshima Prefecture),Kanmuri (JA3 Kyoto Prefecture),Karimo (JA3 Wakayama Prefecture),Kasado (JA4 Yamaguchi Prefecture),Kasasa (JA4 Yamaguchi Prefecture),Kashira (JA4 Okayama Prefecture),Ke (JA3 Kyoto Prefecture),Kosagi (JA4 Hiroshima Prefecture),Kou (JA4 Okayama Prefecture),Kurakake (JA4 Yamaguchi Prefecture),Kuro (JA3 Wakayama Prefecture),Kuro (JA4 Okayama Prefecture),Kuro (JA4 Yamaguchi Prefecture),Kurokami (JA4 Yamaguchi Prefecture),Mae (JA4 Yamaguchi Prefecture),Manabe (JA4 Okayama Prefecture),Me (JA4 Yamaguchi Prefecture),Mi (JA4 Yamaguchi Prefecture),Miya (JA4 Hiroshima Prefecture),Momo (JA4 Hiroshima Prefecture),Mu (JA4 Okayama Prefecture),Mutsure (JA4 Yamaguchi Prefecture),Naga (JA4 Hiroshima Prefecture),Nanatsu (JA9 Ishikawa Prefecture),Nino (JA4 Hiroshima Prefecture),Nishi (JA3 Hyogo Prefecture),No (JA4 Yamaguchi Prefecture),Nomi (JA4 Hiroshima Prefecture),Noto (JA9 Ishikawa Prefecture),Nu (JA3 Hyogo Prefecture),O (JA3 Wakayama Prefecture),O (JA4 Yamaguchi Prefecture),O (JA4 Yamaguchi Prefecture),O (JA4 Yamaguchi Prefecture),Obi (JA4 Okayama Prefecture),Okikamuro (JA4 Yamaguchi Prefecture),Okino (JA3 Wakayama Prefecture),Okuno (JA4 Hiroshima Prefecture),Okurokami (JA4 Hiroshima Prefecture),Ominase (JA4 Yamaguchi Prefecture),Osakikami (JA4 Hiroshima Prefecture),Osakishimo (JA4 Hiroshima Prefecture),Oshiba (JA4 Hiroshima Prefecture),Otabu (JA4 Okayama Prefecture),Otsu (JA4 Yamaguchi Prefecture),Saba (JA4 Yamaguchi Prefecture),Sagi (JA4 Hiroshima Prefecture),Sago (JA4 Yamaguchi Prefecture),Sen (JA4 Yamaguchi Prefecture),Shimoguro (JA4 Hiroshima Prefecture),Shimokamagari (JA4 Hiroshima Prefecture),Shimomizu (JA4 Okayama Prefecture),So (JA4 Okayama Prefecture),Ta (JA4 Hiroshima Prefecture),Taka (JA3 Wakayama Prefecture),Taka (JA4 Okayama Prefecture),Taka (JA4 Shimane Prefecture),Take (JA4 Yamaguchi Prefecture),Tanga (JA3 Hyogo Prefecture),Te (JA4 Yamaguchi Prefecture),Toge (JA4 Hiroshima Prefecture),Tomoga (JA3 Wakayama Prefecture),Toyo (JA4 Hiroshima Prefecture),Tsuno (JA4 Yamaguchi Prefecture),Tsuru (JA4 Okayama Prefecture),U (JA4 Yamaguchi Prefecture),Uji (JA4 Hiroshima Prefecture),Uka (JA4 Yamaguchi Prefecture),Uma (JA4 Shimane Prefecture),Uma (JA4 Yamaguchi Prefecture),Ya (JA4 Yamaguchi Prefecture),Yashiro (JA4 Yamaguchi Prefecture),Yoko (JA4 Hiroshima Prefecture),Ieshima Archipelago (JA3 Hyogo Prefecture)


AS-206

Comment
there are no comments for this entry

Islands
Aji,Ashi,Awa,Eno,Hatsu,Himaka,Hime,Hira,Izu,Kaji,Kami,Kasagai,Kinkasan,Kyuroku,Mikomoto,No (JA2 Aichi Prefecture),No (JA7 Miyagi Prefecture),O (JA2 Aichi Prefecture),O (JA2 Mie Prefecture),O (JA7 Iwate Prefecture),O (JA7 Miyagi Prefecture),Oki,Sabaru,Sado,Saku,Saru,Shino,Suga,Suzu,Tashiro,Tobi,Toshi,Tsubaki,Tsukumi (JA2 Aichi Prefecture),Uki (JA1 Chiba Prefecture),Ushi,Yakei


AS-207

Comment
there are no comments for this entry

Islands
Kosa Dvukh Pilotov

WA9PIE

2020-08-03 03:12

administrator   ~0009843

There changes were tested and verified.

Issue History

Date Modified Username Field Change
2018-10-30 08:05 PD9FER New Issue
2018-10-31 16:38 WA9PIE Additional Information Updated View Revisions
2018-10-31 16:38 WA9PIE Assigned To => PD9FER
2018-10-31 16:38 WA9PIE Status new => assigned
2019-02-15 11:02 WA9PIE Note Added: 0007401
2019-02-15 11:02 WA9PIE Project 1 - Backlog => 3 - Current Dev List
2019-02-15 11:02 WA9PIE Assigned To PD9FER => K7ZCZ
2019-03-23 07:26 KB3NPH Note Added: 0007724
2019-04-09 11:18 K7ZCZ Note Added: 0007845
2019-04-09 11:18 K7ZCZ Assigned To K7ZCZ => WA9PIE
2019-04-09 12:05 K7ZCZ Note Added: 0007846
2019-04-11 05:18 K7ZCZ Note Added: 0007847
2019-05-10 18:31 WA9PIE Note Added: 0007929
2019-05-12 20:52 K7ZCZ Note Added: 0007931
2019-05-13 12:57 K7ZCZ Note Added: 0007932
2019-05-13 12:58 K7ZCZ Note Added: 0007933
2019-05-13 12:59 K7ZCZ Note Added: 0007934
2019-06-16 17:38 WA9PIE Project 3 - Current Dev List => 2 - Next Dev List (Holding Area)
2019-09-14 02:38 WA9PIE Project 2 - Next Dev List (Holding Area) => 4 - Business Priorities
2019-11-29 04:27 PD9FER Note Added: 0009479
2019-12-02 00:24 WA9PIE File Added: response.json
2019-12-02 00:24 WA9PIE Note Added: 0009484
2019-12-02 00:25 WA9PIE Assigned To WA9PIE => DOUG
2019-12-05 17:49 WA9PIE Note Added: 0009488
2019-12-05 17:50 WA9PIE Note Edited: 0009488 View Revisions
2020-02-04 05:58 KB3NPH Note Added: 0009579
2020-04-08 05:45 PD9FER Note Added: 0009662
2020-07-02 02:23 WA9PIE Project 4 - Business Priorities => 3 - Current Dev List
2020-07-11 22:06 DOUG Status assigned => resolved
2020-07-11 22:06 DOUG Resolution open => fixed
2020-07-11 22:06 DOUG Note Added: 0009742
2020-07-12 20:25 WA9PIE Fixed in Version => 6.7.0.290
2020-07-13 03:46 vk2byi Note Added: 0009758
2020-07-13 03:48 vk2byi Note Added: 0009759
2020-07-13 03:51 vk2byi Note Added: 0009762
2020-07-13 03:52 vk2byi Note Added: 0009764
2020-07-13 03:57 g3ucq Note Added: 0009766
2020-07-13 05:06 WA9PIE Status resolved => closed
2020-07-13 05:06 WA9PIE Testing Not Started => Beta Successful
2020-07-13 05:06 WA9PIE Note Added: 0009781
2020-07-13 05:30 WA9PIE Status closed => assigned
2020-07-13 05:30 WA9PIE Resolution fixed => reopened
2020-07-13 05:30 WA9PIE Testing Beta Successful => Beta Failed
2020-07-13 05:30 WA9PIE Note Added: 0009785
2020-07-13 19:49 vk2byi Note Added: 0009792
2020-07-13 20:05 WA9PIE File Added: iota2-json.xlsx
2020-07-13 20:05 WA9PIE Note Added: 0009793
2020-07-13 20:13 vk2byi Note Added: 0009794
2020-07-13 20:19 WA9PIE Note Added: 0009795
2020-07-21 06:26 WA9PIE File Added: IOTAchanges.png
2020-07-21 06:26 WA9PIE Note Added: 0009813
2020-07-21 09:17 g3ucq Note Added: 0009816
2020-07-28 03:33 WA9PIE Note Added: 0009825
2020-07-31 11:54 WA9PIE Note Added: 0009837
2020-07-31 14:31 g3ucq Note Added: 0009838
2020-08-01 00:13 WA9PIE Note Added: 0009839
2020-08-01 00:18 WA9PIE File Added: IOTA Specific Changes.txt
2020-08-01 00:18 WA9PIE Note Added: 0009840
2020-08-03 03:12 WA9PIE Fixed in Version 6.7.0.290 => 6.7.0.301
2020-08-03 03:12 WA9PIE Note Added: 0009843
2020-08-03 03:12 WA9PIE Status assigned => closed
2020-08-03 03:14 WA9PIE Project 3 - Current Dev List => Ham Radio Deluxe