D
Deno

help

Set-Cookie - Header shows as "deleted"

Hherkulessi11/11/2023
I am trying to use deno to script/scrape a website. I can successfully log in to the site (using fetch), however the Headers do seem to get redacted. Instead of seeing a long token, I only get "deleted". the Cookie is set to expire one second after the epoch (is that influences anything) and another Cookie shows up completely fine. Without the cookie I can't take any further actions on the site, because the Cookie in question contains the Session Cookie.
DAHDeno AI Helper11/11/2023
According to the documentation, Deno does not have a cookie jar, which means it does not process or filter the set-cookie header on a response. This could be the reason why you're seeing the set-cookie header as "deleted". In Deno, the set-cookie header is not concatenated when iterated over in the headers iterator. This behavior is in the process of being specified. If you're trying to manage sessions or cookies, you might need to handle them manually in your code. For example, you could parse the set-cookie header from the response and store the session cookie in a variable. Then, you could include this session cookie in the cookie header of your subsequent requests. Please note that this advice is not explicitly stated in the documentation, but is a general approach to handling cookies in environments where they are not automatically managed. 🦕
Hherkulessi11/11/2023
I do know I have to manually handle cookies, which is a bit annoying, but fine for my usecase. What I don't understand is why I don't get the actual cookie value when I read the get-cookies Header from the response, but a literal deleted.
DAHDeno AI Helper11/11/2023
The Set-Cookie header showing as "deleted" could be due to the server setting the cookie to be deleted. This is usually done by setting the cookie's expiration date to a time in the past. This effectively tells the browser (or in this case, your Deno script) to delete the cookie. If the server is setting the cookie to be deleted immediately after setting it, it could be due to a variety of reasons. For example, it could be a mechanism to prevent session fixation attacks, or it could be due to some server-side error. Without more information about the server's behavior or the specific website you're trying to scrape, it's hard to say for sure. You might want to check the server's behavior in a web browser to see if the same thing happens there. If it does, it's likely a server-side issue. If it doesn't, it could be an issue with how Deno is handling the Set-Cookie header. Remember that web scraping should be done responsibly and in accordance with the website's terms of service. Some websites may employ mechanisms to prevent or limit web scraping.
Hherkulessi11/11/2023
A Browser shows the Header as expiring with the current session, however when using curl the set-cookie contains a expiry date at the unix epoch. I will have to test further whether the browser or deno is misbehaving or if it's the server.
DAHDeno AI Helper11/11/2023
Based on the documentation, it's important to note that Deno does not have a cookie jar, which means it does not process or filter the set-cookie header on a response. This could potentially explain the discrepancy you're seeing between the browser and Deno. The set-cookie header showing as "deleted" and the expiry date being set to the Unix epoch (which is a common way to delete a cookie) could be due to the server setting the cookie to be deleted. This is usually done by setting the cookie's expiration date to a time in the past. If the server is setting the cookie to be deleted immediately after setting it, it could be due to a variety of reasons. For example, it could be a mechanism to prevent session fixation attacks, or it could be due to some server-side error. Without more information about the server's behavior or the specific website you're trying to scrape, it's hard to say for sure. You might want to check the server's behavior in a web browser to see if the same thing happens there. If it does, it's likely a server-side issue. If it doesn't, it could be an issue with how Deno is handling the Set-Cookie header. Remember that web scraping should be done responsibly and in accordance with the website's terms of service. Some websites may employ mechanisms to prevent or limit web scraping.
LLeokuma11/12/2023
If it helps in any way, I've been working on Curl bindings for Deno, but it hasn't been tested on many systems. It should not work in Ubuntu 20 or lower for example In order to activate Curl's cookies engine, you set the cookie file to an empty string, like: decurl.setCookiefile(''). That's how libcurl works https://github.com/Leokuma/decurl
GitHub
GitHub - Leokuma/decurl: Curl bindings for Deno.
Curl bindings for Deno. Contribute to Leokuma/decurl development by creating an account on GitHub.
Hherkulessi11/12/2023
That might help but wasn't the issue. In case anybody finds this in the future: If you server returns a redirect, deno seems to follow it. Since deno doesn't handles cookies the Cookie was missing in the second request, so the Server wanted to delete the cookie. I only got the final cookie, not the one returned in the redirect.
LLeokuma11/12/2023
MDN Web Docs
Response: redirected property - Web APIs | MDN
The read-only redirected property of the Response interface indicates whether or not the response is the result of a request you made which was redirected.

Looking for more? Join the community!

Recommended Posts
I keep running into python download failed while trying to run 'cargo test' commandI am currently following the steps I. Deno's Contributing Guide and I am running cargo test but I kehow I can create an restful api with deno and firebase as database?how I can create an restful api with deno and firebase as database?How can I replace writeAllSync with the Streams API?How would one best refactor the following code to use the Web Streams API as suggested by the deprecIs possible to run nodejs (typescript) project with esm modules?I'm trying to run small nodejs project with esm modules. Simple example ```ts // foo.ts import { bAny tutorial links to help me understand the Schedule string in Deno.cron()I am currently working on an issue on the deno repo and I am trying to reconstruct the deno scheduleDeno Deploy failed:Uncaught Exception /Permission Denied : Deno.exit() is not allowedI have a college assignments that needs to be deployed using deno deploy but i get this message whenConnect to a WebSocket through a HTTP(S)/SOCKS5 proxyTitle - I am trying to connect to a `wss://` websocket through a proxy of choice at runtime. Is therExternal package code throws error: window is undefinedHello, I have imported an npm package to my project. And in this package's code, there is a conditiError with importsI have this fancy peace of code that import a bunch of files from a directory: ```ts const utils = Deno.env via fileHow can I use set env from a file and get them via deno.env.get() function as i want to push my codeRunning LLaMA on DenoHello! I use Deno to run my apps on the edge, what Deno solution or other provider would be ideal Occasional very slow isolate start timesI have a Vue.js app (using Vite) project with a few edge functions running on Deno Deploy. What I'mChanging terminal zoom in windows?Is it possible to change the terminal zoom using deno? Also, this is about windows terminal, not conWhimsical error shows up when I run a fileI'm trying to make a discord bot with Deno and d.js but for some reasons I get this error ```tex errI cannot able to install itwhen i run ```sh irm https://deno.land/install.ps1 | iex ``` i got this error messageDeno lint custom rulesHey is there currently a way to create custom rules for Deno lint? And if so how can I specify a cususing OpenAI npm in Deno deploy playground:i don't know how to return the resualt , now this just return [object Object] i don't how to use asFrontend & Backend in one Deno Deploy projectI have a Nuxt 3 frontend and an Oak backend. Is it technically possible to deploy both in one deno d