text visible in browser but not in source

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

text visible in browser but not in source

fugee ohu
I'm not very good with the consoles in chrome and firefox but I couldn't find the text I was looking for in source even though it's displayed as text seemingly, the cursur changes to a vertical line on mouse-over I found this html below in the source How does this html create the text that displays?

   <div class="ui-box product-description-main" id="j-product-description">
        <div class="ui-box-title">Product Description</div>
        <div class="ui-box-body">

            <div class="description-content" data-role="description" data-spm="1000023">
            <div class="loading32"></div>
            </div>

        </div>
    </div>

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

Colin Law
I should think that javascript is involved.  I am sure you asked a
similar question before when you were trying to scrape a website and
couldn't find the text in the html.

Colin
On Wed, 7 Nov 2018 at 15:35, fugee ohu <[hidden email]> wrote:

>
> I'm not very good with the consoles in chrome and firefox but I couldn't find the text I was looking for in source even though it's displayed as text seemingly, the cursur changes to a vertical line on mouse-over I found this html below in the source How does this html create the text that displays?
>
>    <div class="ui-box product-description-main" id="j-product-description">
>         <div class="ui-box-title">Product Description</div>
>         <div class="ui-box-body">
>
>             <div class="description-content" data-role="description" data-spm="1000023">
>             <div class="loading32"></div>
>             </div>
>
>         </div>
>     </div>
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> To post to this group, send email to [hidden email].
> To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/CAL%3D0gLutoo0y24s95MA4eozwUz9ZhdReHTKJr5EH_Le24tMLRA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

fugee ohu


On Wednesday, November 7, 2018 at 11:01:32 AM UTC-5, Colin Law wrote:
I should think that javascript is involved.  I am sure you asked a
similar question before when you were trying to scrape a website and
couldn't find the text in the html.

Colin
On Wed, 7 Nov 2018 at 15:35, fugee ohu <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="kyAsP3UJBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">fuge...@...> wrote:

>
> I'm not very good with the consoles in chrome and firefox but I couldn't find the text I was looking for in source even though it's displayed as text seemingly, the cursur changes to a vertical line on mouse-over I found this html below in the source How does this html create the text that displays?
>
>    <div class="ui-box product-description-main" id="j-product-description">
>         <div class="ui-box-title">Product Description</div>
>         <div class="ui-box-body">
>
>             <div class="description-content" data-role="description" data-spm="1000023">
>             <div class="loading32"></div>
>             </div>
>
>         </div>
>     </div>
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="kyAsP3UJBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonrails-ta...@googlegroups.com.
> To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="kyAsP3UJBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonra...@googlegroups.com.
> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com.
> For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

 Yes, within that context, javascript, how does it happen that the text I'm viewing in the browser isn't visible in source?

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

Colin Law
On Wed, 7 Nov 2018 at 16:17, fugee ohu <[hidden email]> wrote:
>
>  Yes, within that context, javascript, how does it happen that the text I'm viewing in the browser isn't visible in source?

It isn't in the source, the DOM is updated using javascript.  You
should see it in the DOM inspector but not in the source.

Colin

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/CAL%3D0gLtu-SxVBw%2B_BJy6-YjCsVV53K%2BNC1GZZkNoYsezbU%3Dr0A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

Jake Niemiec
In reply to this post by fugee ohu
The ui-box class would indicate that it is a react component: https://github.com/segmentio/ui-box

React components are run client-side, meaning the text you are looking for is inserted into the document after the page runs <script> tags. I would take a look at the Sources tab in chrome, you can find all the loaded scripts there.

On Wed, Nov 7, 2018 at 10:17 AM fugee ohu <[hidden email]> wrote:


On Wednesday, November 7, 2018 at 11:01:32 AM UTC-5, Colin Law wrote:
I should think that javascript is involved.  I am sure you asked a
similar question before when you were trying to scrape a website and
couldn't find the text in the html.

Colin
On Wed, 7 Nov 2018 at 15:35, fugee ohu <[hidden email]> wrote:

>
> I'm not very good with the consoles in chrome and firefox but I couldn't find the text I was looking for in source even though it's displayed as text seemingly, the cursur changes to a vertical line on mouse-over I found this html below in the source How does this html create the text that displays?
>
>    <div class="ui-box product-description-main" id="j-product-description">
>         <div class="ui-box-title">Product Description</div>
>         <div class="ui-box-body">
>
>             <div class="description-content" data-role="description" data-spm="1000023">
>             <div class="loading32"></div>
>             </div>
>
>         </div>
>     </div>
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> To post to this group, send email to [hidden email].

 Yes, within that context, javascript, how does it happen that the text I'm viewing in the browser isn't visible in source?

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/CALn2xuBszCYvKNbS0cwk851YY2SzuDzbZ%2BT5B%3DoHbD1tDLftqQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

fugee ohu


On Wednesday, November 7, 2018 at 12:28:05 PM UTC-5, Jake Niemiec wrote:
The ui-box class would indicate that it is a react component: <a href="https://github.com/segmentio/ui-box" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fsegmentio%2Fui-box\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNE9ztlEb0fu0_cQ43mhWUe4CbnE-w&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fsegmentio%2Fui-box\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNE9ztlEb0fu0_cQ43mhWUe4CbnE-w&#39;;return true;">https://github.com/segmentio/ui-box

React components are run client-side, meaning the text you are looking for is inserted into the document after the page runs <script> tags. I would take a look at the Sources tab in chrome, you can find all the loaded scripts there.

On Wed, Nov 7, 2018 at 10:17 AM fugee ohu <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="Twu_gC4OBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">fuge...@...> wrote:


On Wednesday, November 7, 2018 at 11:01:32 AM UTC-5, Colin Law wrote:
I should think that javascript is involved.  I am sure you asked a
similar question before when you were trying to scrape a website and
couldn't find the text in the html.

Colin
On Wed, 7 Nov 2018 at 15:35, fugee ohu <[hidden email]> wrote:

>
> I'm not very good with the consoles in chrome and firefox but I couldn't find the text I was looking for in source even though it's displayed as text seemingly, the cursur changes to a vertical line on mouse-over I found this html below in the source How does this html create the text that displays?
>
>    <div class="ui-box product-description-main" id="j-product-description">
>         <div class="ui-box-title">Product Description</div>
>         <div class="ui-box-body">
>
>             <div class="description-content" data-role="description" data-spm="1000023">
>             <div class="loading32"></div>
>             </div>
>
>         </div>
>     </div>
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-ta...@googlegroups.com.
> To post to this group, send email to [hidden email].
> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com.
> For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

 Yes, within that context, javascript, how does it happen that the text I'm viewing in the browser isn't visible in source?

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="Twu_gC4OBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonrails-ta...@googlegroups.com.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="Twu_gC4OBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonra...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

Thanks Can you point me to a brief tutorial to show me how to get react to render the content 

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/ba79beeb-e4d3-4808-bc60-a0a76d0b68bf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

Colin Law
On Thu, 8 Nov 2018 at 06:09, fugee ohu <[hidden email]> wrote:
> ...
> Thanks Can you point me to a brief tutorial to show me how to get react to render the content

Open it in a browser, that's what browsers do.

Note there may well be successive requests back to the server to get
the data you are looking for. Look at the Network tab in the browser
developer tools and you may see the call that fetches it.

Colin

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/CAL%3D0gLsPEz6hLp5yOngJVjAOBuWJ2ye0AuSZO%3D15JFyPuyB-iw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

fugee ohu
In reply to this post by Jake Niemiec


On Wednesday, November 7, 2018 at 12:28:05 PM UTC-5, Jake Niemiec wrote:
The ui-box class would indicate that it is a react component: <a href="https://github.com/segmentio/ui-box" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fsegmentio%2Fui-box\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNE9ztlEb0fu0_cQ43mhWUe4CbnE-w&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fsegmentio%2Fui-box\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNE9ztlEb0fu0_cQ43mhWUe4CbnE-w&#39;;return true;">https://github.com/segmentio/ui-box

React components are run client-side, meaning the text you are looking for is inserted into the document after the page runs <script> tags. I would take a look at the Sources tab in chrome, you can find all the loaded scripts there.

On Wed, Nov 7, 2018 at 10:17 AM fugee ohu <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="Twu_gC4OBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">fuge...@...> wrote:


On Wednesday, November 7, 2018 at 11:01:32 AM UTC-5, Colin Law wrote:
I should think that javascript is involved.  I am sure you asked a
similar question before when you were trying to scrape a website and
couldn't find the text in the html.

Colin
On Wed, 7 Nov 2018 at 15:35, fugee ohu <[hidden email]> wrote:

>
> I'm not very good with the consoles in chrome and firefox but I couldn't find the text I was looking for in source even though it's displayed as text seemingly, the cursur changes to a vertical line on mouse-over I found this html below in the source How does this html create the text that displays?
>
>    <div class="ui-box product-description-main" id="j-product-description">
>         <div class="ui-box-title">Product Description</div>
>         <div class="ui-box-body">
>
>             <div class="description-content" data-role="description" data-spm="1000023">
>             <div class="loading32"></div>
>             </div>
>
>         </div>
>     </div>
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-ta...@googlegroups.com.
> To post to this group, send email to [hidden email].
> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com.
> For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

 Yes, within that context, javascript, how does it happen that the text I'm viewing in the browser isn't visible in source?

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="Twu_gC4OBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonrails-ta...@googlegroups.com.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="Twu_gC4OBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonra...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

I was able to find the text that wasn't shown in source by opening console and expanding the ui-box div 

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/f4193c37-b0a4-4dba-9f99-b2a1db258fca%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

fugee ohu
In reply to this post by Jake Niemiec


On Wednesday, November 7, 2018 at 12:28:05 PM UTC-5, Jake Niemiec wrote:
The ui-box class would indicate that it is a react component: <a href="https://github.com/segmentio/ui-box" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fsegmentio%2Fui-box\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNE9ztlEb0fu0_cQ43mhWUe4CbnE-w&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fsegmentio%2Fui-box\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNE9ztlEb0fu0_cQ43mhWUe4CbnE-w&#39;;return true;">https://github.com/segmentio/ui-box

React components are run client-side, meaning the text you are looking for is inserted into the document after the page runs <script> tags. I would take a look at the Sources tab in chrome, you can find all the loaded scripts there.

On Wed, Nov 7, 2018 at 10:17 AM fugee ohu <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="Twu_gC4OBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">fuge...@...> wrote:


On Wednesday, November 7, 2018 at 11:01:32 AM UTC-5, Colin Law wrote:
I should think that javascript is involved.  I am sure you asked a
similar question before when you were trying to scrape a website and
couldn't find the text in the html.

Colin
On Wed, 7 Nov 2018 at 15:35, fugee ohu <[hidden email]> wrote:

>
> I'm not very good with the consoles in chrome and firefox but I couldn't find the text I was looking for in source even though it's displayed as text seemingly, the cursur changes to a vertical line on mouse-over I found this html below in the source How does this html create the text that displays?
>
>    <div class="ui-box product-description-main" id="j-product-description">
>         <div class="ui-box-title">Product Description</div>
>         <div class="ui-box-body">
>
>             <div class="description-content" data-role="description" data-spm="1000023">
>             <div class="loading32"></div>
>             </div>
>
>         </div>
>     </div>
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-ta...@googlegroups.com.
> To post to this group, send email to [hidden email].
> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com.
> For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

 Yes, within that context, javascript, how does it happen that the text I'm viewing in the browser isn't visible in source?

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="Twu_gC4OBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonrails-ta...@googlegroups.com.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="Twu_gC4OBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonra...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

So far I'm trying to get up to the table, the last element shown below   doc.at_css("div#j-product-description div.ui-box-body div.description-content") gets me back the div class="description-content element but  doc.at_css("div#j-product-description div.ui-box-body div.description-content div.origin-part") returns nil There's a lot inside kde:widget that I'm not including here

<div class="ui-box product-description-main" id="j-product-description" data-widget-cid="widget-27">
        <div class="ui-box-title">Product Description</div>
        <div class="ui-box-body">
<div class="description-content" data-role="description" data-spm="1000023"><div class="origin-part"><p> <br> <br> <br> &nbsp; </p> 
<kse:widget data-widget-type="relatedProduct" id="24226336" title="TOP" type="relation">...</kse:widget> 
<table border="2">

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/34f70d1c-c808-4ac1-9115-fa6bf22f82c5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

Walter Lee Davis

> On Nov 9, 2018, at 6:22 PM, fugee ohu <[hidden email]> wrote:
>
>
>
> On Wednesday, November 7, 2018 at 12:28:05 PM UTC-5, Jake Niemiec wrote:
> The ui-box class would indicate that it is a react component: https://github.com/segmentio/ui-box
>
> React components are run client-side, meaning the text you are looking for is inserted into the document after the page runs <script> tags. I would take a look at the Sources tab in chrome, you can find all the loaded scripts there.
>
> On Wed, Nov 7, 2018 at 10:17 AM fugee ohu <[hidden email]> wrote:
>
>
> On Wednesday, November 7, 2018 at 11:01:32 AM UTC-5, Colin Law wrote:
> I should think that javascript is involved.  I am sure you asked a
> similar question before when you were trying to scrape a website and
> couldn't find the text in the html.
>
> Colin
> On Wed, 7 Nov 2018 at 15:35, fugee ohu <[hidden email]> wrote:
> >
> > I'm not very good with the consoles in chrome and firefox but I couldn't find the text I was looking for in source even though it's displayed as text seemingly, the cursur changes to a vertical line on mouse-over I found this html below in the source How does this html create the text that displays?
> >
> >    <div class="ui-box product-description-main" id="j-product-description">
> >         <div class="ui-box-title">Product Description</div>
> >         <div class="ui-box-body">
> >
> >             <div class="description-content" data-role="description" data-spm="1000023">
> >             <div class="loading32"></div>
> >             </div>
> >
> >         </div>
> >     </div>
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> > To post to this group, send email to [hidden email].
> > To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
>  Yes, within that context, javascript, how does it happen that the text I'm viewing in the browser isn't visible in source?
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> To post to this group, send email to [hidden email].
> To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
> So far I'm trying to get up to the table, the last element shown below   doc.at_css("div#j-product-description div.ui-box-body div.description-content") gets me back the div class="description-content element but  doc.at_css("div#j-product-description div.ui-box-body div.description-content div.origin-part") returns nil There's a lot inside kde:widget that I'm not including here
>
> <div class="ui-box product-description-main" id="j-product-description" data-widget-cid="widget-27">
>         <div class="ui-box-title">Product Description</div>
>         <div class="ui-box-body">
> <div class="description-content" data-role="description" data-spm="1000023"><div class="origin-part"><p> <br> <br> <br> &nbsp; </p>
> <kse:widget data-widget-type="relatedProduct" id="24226336" title="TOP" type="relation">...</kse:widget>
> <table border="2">
>

It seems to me that you are going to have to identify the data source that the in-page JavaScript is using to generate the dynamic table data, and query that rather than trying to work everything out from the HTML (which is just a template for the in-page script to fill). There's probably a JSON URL somewhere that is being loaded into the page, and the script is building from that. This entire approach is pretty fraught with peril, though, because (like any scraping project, only more so) any change to the scheme that the site's developer chooses to implement will break your scraper immediately.

Following this path is going to force you to learn about how the site is working on a code level -- and to figure out how they go from data to presentation.

Another approach might be to use a headless browser on the server to construct a "real" DOM of the page, and query that. To be clear -- I do not recommend you follow this path -- I am noting it here to illustrate how ridiculous this effort will be.

One way to visualize this difference is to use the Web Inspector in Safari or Chrome to look at the differences between the raw HTML (Safari labels this tab "Resources") and the DOM (Safari calls this "Elements"). There is likely very little in common outside of the overall outline, if the page is changing as dramatically as you describe. If you hunt through the Resources tab (in Safari) you may find a link to a JSON file that is being required into the page. Loading that URL, rather than the HTML, may give you a much cleaner set of data (which you can parse directly using Ruby) rather than trying to execute JS on your server in order to construct an HTML DOM that you can parse with Nokogiri.

Walter

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/009F2585-34BA-4B4D-A0F0-AD561978E613%40wdstudio.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

fugee ohu


On Saturday, November 10, 2018 at 10:35:03 AM UTC-5, Walter Lee Davis wrote:

> On Nov 9, 2018, at 6:22 PM, fugee ohu <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="1i-UEsHzBQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">fuge...@...> wrote:
>
>
>
> On Wednesday, November 7, 2018 at 12:28:05 PM UTC-5, Jake Niemiec wrote:
> The ui-box class would indicate that it is a react component: <a href="https://github.com/segmentio/ui-box" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fsegmentio%2Fui-box\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNE9ztlEb0fu0_cQ43mhWUe4CbnE-w&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fsegmentio%2Fui-box\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNE9ztlEb0fu0_cQ43mhWUe4CbnE-w&#39;;return true;">https://github.com/segmentio/ui-box
>
> React components are run client-side, meaning the text you are looking for is inserted into the document after the page runs <script> tags. I would take a look at the Sources tab in chrome, you can find all the loaded scripts there.
>
> On Wed, Nov 7, 2018 at 10:17 AM fugee ohu <[hidden email]> wrote:
>
>
> On Wednesday, November 7, 2018 at 11:01:32 AM UTC-5, Colin Law wrote:
> I should think that javascript is involved.  I am sure you asked a
> similar question before when you were trying to scrape a website and
> couldn't find the text in the html.
>
> Colin
> On Wed, 7 Nov 2018 at 15:35, fugee ohu <[hidden email]> wrote:
> >
> > I'm not very good with the consoles in chrome and firefox but I couldn't find the text I was looking for in source even though it's displayed as text seemingly, the cursur changes to a vertical line on mouse-over I found this html below in the source How does this html create the text that displays?
> >
> >    <div class="ui-box product-description-main" id="j-product-description">
> >         <div class="ui-box-title">Product Description</div>
> >         <div class="ui-box-body">
> >
> >             <div class="description-content" data-role="description" data-spm="1000023">
> >             <div class="loading32"></div>
> >             </div>
> >
> >         </div>
> >     </div>
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-ta...@googlegroups.com.
> > To post to this group, send email to [hidden email].
> > To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/rubyonrails-talk/8e0eb26a-517a-4216-bb9c-8bd05e4412a5%40googlegroups.com.
> > For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.
>
>  Yes, within that context, javascript, how does it happen that the text I'm viewing in the browser isn't visible in source?
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-ta...@googlegroups.com.
> To post to this group, send email to [hidden email].
> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/rubyonrails-talk/12b65225-60e5-4fe3-80a7-9ebb8013f312%40googlegroups.com.
> For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.
>
> So far I'm trying to get up to the table, the last element shown below   doc.at_css("div#j-product-description div.ui-box-body div.description-content") gets me back the div class="description-content element but  doc.at_css("div#j-product-description div.ui-box-body div.description-content div.origin-part") returns nil There's a lot inside kde:widget that I'm not including here
>
> <div class="ui-box product-description-main" id="j-product-description" data-widget-cid="widget-27">
>         <div class="ui-box-title">Product Description</div>
>         <div class="ui-box-body">
> <div class="description-content" data-role="description" data-spm="1000023"><div class="origin-part"><p> <br> <br> <br> &nbsp; </p>
> <kse:widget data-widget-type="relatedProduct" id="24226336" title="TOP" type="relation">...</kse:widget>
> <table border="2">
>

It seems to me that you are going to have to identify the data source that the in-page JavaScript is using to generate the dynamic table data, and query that rather than trying to work everything out from the HTML (which is just a template for the in-page script to fill). There's probably a JSON URL somewhere that is being loaded into the page, and the script is building from that. This entire approach is pretty fraught with peril, though, because (like any scraping project, only more so) any change to the scheme that the site's developer chooses to implement will break your scraper immediately.

Following this path is going to force you to learn about how the site is working on a code level -- and to figure out how they go from data to presentation.

Another approach might be to use a headless browser on the server to construct a "real" DOM of the page, and query that. To be clear -- I do not recommend you follow this path -- I am noting it here to illustrate how ridiculous this effort will be.

One way to visualize this difference is to use the Web Inspector in Safari or Chrome to look at the differences between the raw HTML (Safari labels this tab "Resources") and the DOM (Safari calls this "Elements"). There is likely very little in common outside of the overall outline, if the page is changing as dramatically as you describe. If you hunt through the Resources tab (in Safari) you may find a link to a JSON file that is being required into the page. Loading that URL, rather than the HTML, may give you a much cleaner set of data (which you can parse directly using Ruby) rather than trying to execute JS on your server in order to construct an HTML DOM that you can parse with Nokogiri.

Walter


It wasn't shown in source but when I expanded the element recursively in chrome developer tools I saw the text I was looking for So, what's that gonna be worth?

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/4d5c228f-5252-46b4-9ab0-72257d754ead%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

Colin Law
On Sat, 10 Nov 2018 at 17:23, fugee ohu <[hidden email]> wrote:
> ...
> It wasn't shown in source but when I expanded the element recursively in chrome developer tools I saw the text I was looking for So, what's that gonna be worth?

As has been said a number of times that will be because it was filled
in by javascript, probably as a result of further calls to the server.

Colin

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/CAL%3D0gLudDPm3UoJAdmqiOcyH1obDrQ-3CBkn4sMW5nLD_LaFfQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

fugee ohu


On Saturday, November 10, 2018 at 12:26:40 PM UTC-5, Colin Law wrote:
On Sat, 10 Nov 2018 at 17:23, fugee ohu <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="yEXoVdj5BQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">fuge...@...> wrote:
> ...
> It wasn't shown in source but when I expanded the element recursively in chrome developer tools I saw the text I was looking for So, what's that gonna be worth?

As has been said a number of times that will be because it was filled
in by javascript, probably as a result of further calls to the server. 

Colin

Using a headless browser would be cheating?

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

Colin Law
On Sat, 10 Nov 2018 at 21:28, fugee ohu <[hidden email]> wrote:

>
>
>
> On Saturday, November 10, 2018 at 12:26:40 PM UTC-5, Colin Law wrote:
>>
>> On Sat, 10 Nov 2018 at 17:23, fugee ohu <[hidden email]> wrote:
>> > ...
>> > It wasn't shown in source but when I expanded the element recursively in chrome developer tools I saw the text I was looking for So, what's that gonna be worth?
>>
>> As has been said a number of times that will be because it was filled
>> in by javascript, probably as a result of further calls to the server.

Have you done what I suggested and looked in the browser developer
tools at the Network tab?  Then you will see if it fetches any further
data after the initial page fetch.  Very often you will find it
fetching some json which will very likely contain the data you are
looking for.

Colin

>>
>> Colin
>
>
> Using a headless browser would be cheating?
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> To post to this group, send email to [hidden email].
> To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/CAL%3D0gLvTG-F0nZpjhm%3DGSP7Y8ZT-pj9gV7CLBew6%3DMDS_8igVQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

fugee ohu


On Saturday, November 10, 2018 at 4:57:54 PM UTC-5, Colin Law wrote:
On Sat, 10 Nov 2018 at 21:28, fugee ohu <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="tp4EbqUIBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">fuge...@...> wrote:

>
>
>
> On Saturday, November 10, 2018 at 12:26:40 PM UTC-5, Colin Law wrote:
>>
>> On Sat, 10 Nov 2018 at 17:23, fugee ohu <[hidden email]> wrote:
>> > ...
>> > It wasn't shown in source but when I expanded the element recursively in chrome developer tools I saw the text I was looking for So, what's that gonna be worth?
>>
>> As has been said a number of times that will be because it was filled
>> in by javascript, probably as a result of further calls to the server.

Have you done what I suggested and looked in the browser developer
tools at the Network tab?  Then you will see if it fetches any further
data after the initial page fetch.  Very often you will find it
fetching some json which will very likely contain the data you are
looking for.

Colin

>>
>> Colin
>
>
> Using a headless browser would be cheating?
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="tp4EbqUIBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonrails-ta...@googlegroups.com.
> To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="tp4EbqUIBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonra...@googlegroups.com.
> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com.
> For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

 In chrome developer tools under network->all there's a lot of config.json?key=.... statements but under I don't see anything

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/054eba47-d1ee-43c3-b10e-410ab8fac54f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

fugee ohu
In reply to this post by Colin Law


On Saturday, November 10, 2018 at 4:57:54 PM UTC-5, Colin Law wrote:
On Sat, 10 Nov 2018 at 21:28, fugee ohu <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="tp4EbqUIBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">fuge...@...> wrote:

>
>
>
> On Saturday, November 10, 2018 at 12:26:40 PM UTC-5, Colin Law wrote:
>>
>> On Sat, 10 Nov 2018 at 17:23, fugee ohu <[hidden email]> wrote:
>> > ...
>> > It wasn't shown in source but when I expanded the element recursively in chrome developer tools I saw the text I was looking for So, what's that gonna be worth?
>>
>> As has been said a number of times that will be because it was filled
>> in by javascript, probably as a result of further calls to the server.

Have you done what I suggested and looked in the browser developer
tools at the Network tab?  Then you will see if it fetches any further
data after the initial page fetch.  Very often you will find it
fetching some json which will very likely contain the data you are
looking for.

Colin

>>
>> Colin
>
>
> Using a headless browser would be cheating?
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="tp4EbqUIBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonrails-ta...@googlegroups.com.
> To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="tp4EbqUIBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonra...@googlegroups.com.
> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com.
> For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

Yes there's a few scripts running on the page

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/0b5a2859-e95d-4b5d-85d6-17011939617b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: text visible in browser but not in source

fugee ohu
In reply to this post by Colin Law


On Saturday, November 10, 2018 at 4:57:54 PM UTC-5, Colin Law wrote:
On Sat, 10 Nov 2018 at 21:28, fugee ohu <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="tp4EbqUIBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">fuge...@...> wrote:

>
>
>
> On Saturday, November 10, 2018 at 12:26:40 PM UTC-5, Colin Law wrote:
>>
>> On Sat, 10 Nov 2018 at 17:23, fugee ohu <[hidden email]> wrote:
>> > ...
>> > It wasn't shown in source but when I expanded the element recursively in chrome developer tools I saw the text I was looking for So, what's that gonna be worth?
>>
>> As has been said a number of times that will be because it was filled
>> in by javascript, probably as a result of further calls to the server.

Have you done what I suggested and looked in the browser developer
tools at the Network tab?  Then you will see if it fetches any further
data after the initial page fetch.  Very often you will find it
fetching some json which will very likely contain the data you are
looking for.

Colin

>>
>> Colin
>
>
> Using a headless browser would be cheating?
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="tp4EbqUIBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonrails-ta...@googlegroups.com.
> To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="tp4EbqUIBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rubyonra...@googlegroups.com.
> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/rubyonrails-talk/a9b07a26-4caf-450b-b436-11c9762a7456%40googlegroups.com.
> For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

I found this under Network->Initiator The name column to the left of Initiator is very long I got these from mouseover one of the Initiators

 send @ package.a6067778.js:3
ajax @ package.a6067778.js:3
_init @ main-detail-v170105.f7571793.js:24
setup @ main-detail-v170105.f7571793.js:24
initialize @ package.a6067778.js:4
o @ package.a6067778.js:4
getTrendingProductFun @ main-detail-v170105.f7571793.js:25
onRouse @ main-detail-v170105.f7571793.js:25
_rouse @ main-detail-v170105.f7571793.js:13
_activate @ main-detail-v170105.f7571793.js:13
activateAll @ main-detail-v170105.f7571793.js:13
(anonymous) @ main-detail-v170105.f7571793.js:13
(anonymous) @ ZQ8V5-LYZLD-DEX8D-M5HHU-ERM4X:16

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/ca04e9fc-e968-4bd2-9c68-790e4f23261b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.