# Handling redirect when importing from live URL

**URL:** <https://forum.openrefine.org/t/handling-redirect-when-importing-from-live-url/2159>\
**Category:** Support and Helpdesk\
**Created:** [March 18, 2025, 7:57pm UTC](https://forum.openrefine.org/t/handling-redirect-when-importing-from-live-url/2159 "2025-03-18T19:57:56Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![northern-hills](https://avatars.discourse-cdn.com/v4/letter/n/ed655f/32.png) [@northern-hills](https://forum.openrefine.org/u/northern-hills)\
**Post date:** [March 18, 2025, 7:57pm UTC](https://forum.openrefine.org/t/handling-redirect-when-importing-from-live-url/2159/1 "2025-03-18T19:57:56Z")

</div>

Hello! I'm trying to pull data into an OpenRefine project from a CSV export on an open data portal. The URL to the CSV redirects to a signed S3 URL with expiring credentials, but OpenRefine doesn't appear to be following the redirect when I use the "Import from Web Addresses" option.

Here's the URL: [https://data.boston.gov/dataset/e63a37e1-be79-4722-89e6-9e7e2a3da6d1/resource/73c7e069-701f-4910-986d-b950f46c91a1/download/tmp3fykdojs.csv](https://data.boston.gov/dataset/e63a37e1-be79-4722-89e6-9e7e2a3da6d1/resource/73c7e069-701f-4910-986d-b950f46c91a1/download/tmp3fykdojs.csv)

... which redirects to a URL like this, with credentials that expire after 24 hours:

[https://s3.amazonaws.com/og-production-open-data-bostonma-892364687672/resources/73c7e069-701f-4910-986d-b950f46c91a1/tmp3fykdojs.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJJIENTAPKHZMIPXQ%2F20250318%2Fus-east-1%2Fs3%2Faws4\_request&X-Amz-Date=20250318T184820Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=3851cad29b5bda96d901f33f4d8c0d2f6c44839c70dc8c29c9569e29239bdfcc](https://s3.amazonaws.com/og-production-open-data-bostonma-892364687672/resources/73c7e069-701f-4910-986d-b950f46c91a1/tmp3fykdojs.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJJIENTAPKHZMIPXQ%2F20250318%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250318T184820Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=3851cad29b5bda96d901f33f4d8c0d2f6c44839c70dc8c29c9569e29239bdfcc)

Is there any way around this problem other than setting up a routine to download the file locally each time?

Thank you! Appreciate all the work everyone has put into this tool, use it all the time.

---

<div class="post-metadata">

**Author:** ![ostephens](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.openrefine.org/ostephens/32/8_2.png) [@ostephens](https://forum.openrefine.org/u/ostephens)\
**Post date:** [March 20, 2025, 10:47am UTC](https://forum.openrefine.org/t/handling-redirect-when-importing-from-live-url/2159/2 "2025-03-20T10:47:29Z")

</div>

TL/DR: I can't get this to work, and there is odd behaviour, it maybe worth [raising a Github issue](https://github.com/OpenRefine/OpenRefine/issues/new/choose) for a developer to dig more deeply

My understanding is that the import from web addresses should follow the redirect automatically. Trying to test the URLs with cURL, I get the redirect fine from the initial URL, but then if I try to cURL the resulting S3 URL I get a 403 response which doesn't tell us a huge amount - it means we're not authorised to access that URL but it doesn't tell us why - potentially a TLS/SSL issue, or maybe something else

On the otherhand it works fine in the browser.

Trying in OpenRefine, I can't get either URL to work when using "Import from web addresses" - I just get the error `Error uploading data null` in both cases which is unhelpful

If I try with the "Add column by fetching URLs" from inside an existing OpenRefine project the S3 URL works fine, but the original [data.boston.gov](http://data.boston.gov) url gives me an error

```auto
org.apache.hc.client5.http.ClientProtocolException: HTTP error 403 : Forbidden for URL https://data.boston.gov/dataset/e63a37e1-be79-4722-89e6-9e7e2a3da6d1/resource/73c7e069-701f-4910-986d-b950f46c91a1/download/tmp3fykdojs.csv

```

Again - its a 403 response that could cover a multitude of sins - it seems odd to get that for a URL that should be openly available and is just a redirect.

That's as far as I could get easily I'm afraid, so I don't have a solution for you

Owen

---

<div class="post-metadata">

**Author:** ![northern-hills](https://avatars.discourse-cdn.com/v4/letter/n/ed655f/32.png) [@northern-hills](https://forum.openrefine.org/u/northern-hills)\
**Post date:** [March 20, 2025, 3:07pm UTC](https://forum.openrefine.org/t/handling-redirect-when-importing-from-live-url/2159/3 "2025-03-20T15:07:08Z")

</div>

Thanks for taking a look! I think the 403 error is because the URL there is unsigned. I'll open a GitHub issue.